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PREFACE 


HOW  TO  USE  THE  PROJECT  MANAGER’S  GUIDE 

The  Guide  is  organized  into  two  parts.  Part  A (chapters  I to  VIII)  is  a chrono- 
logical guide  to  project  management;  part  B (chapters  IX  to  XXII)  is  a guide  to  the  key 
disciplines  encountered  in  implementing  part  A.  Each  chronological  step  depends  on  the 
previous  steps  and  largely  determines  future  directions,  so  you,  the  user,  are  encouraged 
to  become  familiar  with  part  A in  its  entirety.  Part  B is  intended  to  provide  useful 
advice  even  to  the  specialist  in  the  particular  discipline;  in  using  part  A,  it  will  become 
necessary  to  refer  to  sections  of  part  B.  Although  the  procedures  and  guidance  provided 
conform  to  “the  system,”  efforts  have  been  made  to  describe  the  most  effective  and 
efficient  means  to  each  end;  therefore,  some  of  the  concepts  presented  are  non  traditional 
and  may  be  new  to  the  user. 

Should  questions  or  comments  arise  in  using  this  manual,  please  contact  John 
Townsend  by  mail  or  telephone. 

Commander,  Naval  Ocean  Systems  Center 

Attention  JH  Townsend 

271  Catalina  Blvd 

San  Diego,  CA  92152 

Autovon:  933-7295  Commercial:  (714)  225-7295 

This  guide  is  the  product  of  a continuing  effort;  your  comments  are  welcomed  toward 
improving  its  content  and  utility. 


ADMINISTRATIVE  BACKGROUND 

This  guide  was  produced  by  the  Engineering  Sciences  Department,  Naval  Ocean 
Systems  Center,  under  the  auspices  of  the  Center’s  Low  Cost  Electronics/TELCAM  project. 
Low  Cost  Electronics  is  an  acquisition  R&D  project  which  is  studying  and  making  recom- 
mendations on  ways  to  obtain  better  electronics  equipments  while  lowering  their  total 
costs.  TELCAM  (Telecommunications  Equipment  Low  Cost  Acquisition  Method)  addresses 
the  use  of  existing  commercial  and  military  equipments  in  new  military  applications.  Both 
portions  of  the  project  were  responses  by  the  Center  to  the  Electronics  “X”  study1  and' 
others.  The  project  emphasis  has  been  to  implement  practical  methods  of  acquiring  military 
electronics. 

Numerous  dedicated  people  from  commands  of  the  Navy  Department  and  also  the 
Army  and  Air  Force,  from  offices  of  the  Department  of  Defense,  from  the  defense  indus- 
try, and  from  commercial  industry  have  made  substantial  contributions  to  the  effort  since 
its  inception  in  1973.  Special  thanks  go  to  the  faculty  and  students  of  the  Naval  Post- 
graduate School  for  their  outstanding  support. 

Appendix  C lists  the  organizations  which  have  contributed. 


'“Electronics  X:  A Study  of  Military  Electronics  with  Particular  Reference  to  Cost  and  Reliability,” 
Institute  for  Defense  Analyses  Report  R-195,  January  1974 
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PART  A:  CHRONOLOGICAL  GUIDE 
1.  INTRODUCTION 

THE  PATTERN  OF  SUCCESS 

The  role  of  the  project  manager  is  to  acquire  equipments  which  will  perform  the 
required  functions  at  an  affordable  price  and  by  the  time  they  are  needed.  In  these  days 
of  constrained  budgets,  “affordable”  may  be  defined  as  the  least  total  cost  to  the  govern- 
ment only  with  the  proviso  that  the  functions  to  be  performed  are  worth  that  expenditure. 
Literally  hundreds  of  studies  have  looked  at  defense  acquisitions  over  the  past  30  years. 

Reading  the  conclusions  and  recommendations  from  a 1949  report  is  like  reading  the 
results  of  a 1974  report.  Project  after  project  fails  to  achieve  its  goals.  Each  report  is 
clear;  projects  fail  to  meet  performance  objectives,  overrun  costs  by  1 50%,  and  slip 
schedules  25-50%.  In  the  vast  majority  of  cases,  the  project  goals  were  not  achieved  for 
the  following  reasons:  misspecification  (usually  gross  over-specification  creating  artificial 
technical  problems),  failure  to  manage  risks,  obscuring  of  the  project  goals  through 
extraneous  paperwork  requirements,  and  failure  to  adequately  define  what  is  required. 

These  reasons  are,  however,  only  the  symptoms  of  underlying  problems  in  the  acquisition 
community.  The  TELCAM  project  looked  at  successes  and  failures  in  industry  as  well 
as  government;  the  successful  project  has  the  same  traits  whether  in  industry  or  in 
government.  An  acquisition  project  also  shares  many  traits  with  a small  business,  so 
TELCAM  also  solicited  information  from  the  Small  Business  Administration.  Again,  suc- 
cess is  a pattern,  whereas  failure  is  a deviation  from  that  pattern.  The  major  difference 
between  failures  in  industry  and  failures  in  government  is  that  a failing  project  in  industry 
is  usually  rapidly  terminated;  the  government  failure  usually  plods  on  to  an  elegant  wreck. 

What  is  the  elusive  pattern  of  success?  The  projects  cited  as  successes  will  have 
two  main  features: 

• A strong,  knowledgeable  project  manager  who  acts  as  the  ultimate  authority  for 
all  project  matters  — tasking,  budgeting,  technical  decisions. 

• A small,  dedicated  team  executing  project  tasks. 

The  key  words  above  are:  strong,  knowledgeable,  ultimate  authority,  small, 
dedicated,  and  team.  Excellent  studies  of  the  nuclear  power  program,1  the  Polaris  system,2 
and  NTDS3  are  available  which  show  these  forces  at  work.  “Strong”  appears  to  be  a 
peculiar  necessity  in  the  government  projects,  as  each  success  seems  to  attain  that  status  in 
spite  of  “the  system.”  An  ultimate  goal  of  acquisition  R&D  must  be  to  change  “the 
system”  to  allow  average  individuals  to  be  successful  project  managers.  Until  that  goal  is 
reached,  there  is  still  enough  of  a task  to  create  knowledgeable  managers.  Some  spectacu- 
lar failures  have  been  managed  by  strong,  unknowledgeable  individuals.  A project  needs  a * - 

strong  champion  in  order  to  “steal”  sufficient  authority  to  become  a purposeful  auto- 
nomous entity,  but  authority  unwisely  wielded  is  disaster.  The  government  project  man- 
ager is  not  held  accountable  for  his  actions;  accountability  is  the  “quality  assurance” 
incentive  used  to  check  authority  in  industry. 


'Nuclear  Navy  (1946-1962),  RG  Hewlett  and  F Duncan,  Chicago,  1974 
JThe  Polaris  System  Development,  Sapolsky,  Harvard,  1972 
3 The  NTDS  Development,  RW  Graf,  United  Research,  Inc,  29  Jan  1964 
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The  first  procurement  of  muskets  by  the  Army  in  1798  seemed  straightforward. 

Eli  Whitney  promised  to  produce  and  deliver  muskets  built  by  assembly  line  techniques, 
and  using  interchangeable  parts,  within  8 months;  actual  delivery  was  made  10  years  late. 
Our  military  procurement  problems  actually  predate  the  nation  itself;  one  verse  of  “Yankee 
Doodle”  went 


“And  there  we  saw  a thousand  men 
As  rich  as  Squire  David, 

And  what  they  wasted  every  day 
I wish  it  could  be  saved” 


referring  to  one  of  General  Washington’s  encampments. 

On  27  March  1794,  Congress  appropriated  $768  000  for  the  construction  and 
manning  of  six  frigates.  Each  frigate  was  to  cost  $100  000.  When  the  UNITED  STATES, 
CONSTITUTION,  and  CONSTELLATION  were  finally  launched  in  1797,  each  cost  close 
to  $300  000. 

The  innumerable  instances  of  procurement  problems  which  have  occurred  repeat- 
edly over  the  years  have  only  worsened  with  time.  The  evolving  acquisition  system  oper- 
ated in  ignorance  in  the  1 790s,  and  this  basic  ignorance  exists  today  throughout  the 
acquisition  community.  Less  than  30%  of  the  project  managers  interviewed  by  TELCAM 
knew  the  operational  objective  of  their  project;  only  5%  could  relate  technical  features  of 
the  project  to  operational  considerations.  Only  2%  of  the  project  managers  were  aware  of 
any  actions  being  taken  on  their  project  to  reduce  risks  of  any  kind;  only  half  knew  what 
progress  was  1 nade  or  what  difficulties  were  being  encountered  on  major  tasks! 

Under  the-  lances,  it  is  a wonder  even  greater  failures  do  not  occur  than  those 

already  f f the  major  factors  aggravating  this  situation  is  the  lack  of  project 

manage  iy  for  the  end  item  in  the  field.  A project  manager  may  only 

influer  of  project  life,  whereas  the  end  item  may  be  in  use  for  over  20  years. 

The  project  manager  determinations  affect  all  but  a few  percent  ot  the  total  file  costs  of 
a project. 

Figure  1-1  shows  the  percentage  of  total  ownership  costs  committed  during  con- 
ceptual planning,  design,  development,  acquisition,  and  operations  for  past  major  programs. 
In  the  past,  decisions  made  during  the  concept  and  planning  phases  committed  70%  of  the 
total  life-cycle  cost  funds  of  a program,  and  design,  development,  acquisition,  and  opera- 
tions accounted  for  only  30%  of  that  total  cost.  The  effects  of  application  of  life-cycle 
cost  analysis  through  the  planning  and  RDT&E  phases  of  a program,  and  the  “design  to 
cost”  concept  on  new  programs  are  expected  to  change  this  distribution  considerably  by 
affecting  a larger  portion  of  that  early  70%  commitment. 

Notice  that  90%  of  the  total  project  cost  is  fixed  after  only  10%  of  the  funds 
have  been  expended.  Unless  the  project  manager  assumes  responsibility  for  the  total, 
it  is  impossible  to  attain  the  lowest  possible  project  cost.  Considering  that  support  costs 
may  be  altered  by  as  much  as  two  orders  of  magnitude  by  decisions  in  the  conceptual 

' . I _ _ . . » ....  i. ..  J 4 . 


phase,  it  can  be  seen  that  a naive  approach  to  project  management  can  have  a devastating 
effect  on  operating  budgets;  likewise,  sound  management  can  lead  to  a highly  efficient 
use  of  funds. 

The  acquisition  manager  should  try  to  obtain  equipments  which  are  fully  capable 
of  doing  the  required  job  and  which  have  the  following  characteristics: 


I-: 


Supportable 

Procurable 

Producible 

The  remainder  of  the  Guide  discusses  the  many  facets  of  these  factors. 


Figure  1-1.  Systems  funds  committed  by  initial  planning  decisions. 

THE  PROJECT  MANAGER 

A manager  and  a project  are  established  for  a single  purpose.  It  makes  no  difference 
that  the  project  is  designated  a major  program  and  its  manager  a program  manager  or  that 
the  project  is  called  a tasking  with  a project  engineer  in  charge.  Program  manager,  acquisi- 
tion manager,  project  manager  are,  for  the  purposes  of  this  guide,  synonymous,  being  differ- 
ent in  scope  but  like  in  character.  Project  management  is  the  planning,  executing,  directing, 
and  controlling  of  ? relatively  short-term  project  or  systems-oriented  organization  estab- 
lished for  the  completion  of  specific  goals.  In  this  guide,  those  specific  goals  will  be  an 
acquisition  and  implementation  of  military  equipments  and  the  subgoals  associated  with 
each  phase  of  the  cradle-to-grave  life  of  those  equipments;  however,  the  principles  presented 
are  basic  and  universal  and  adaptable  to  many  other  project  circumstances. 

The  project  manager  ideally  will  plan,  organize,  monitor,  and  direct  the  project  to  its 
goals  as  effectively  as  possible.  Efficiency  is  a secondary  consideration,  since  maximum 
efficiency  often  compromises  effectiveness.  It  is  generally  agreed  that,  in  the  competitive 
atmosphere  of  military  affairs,  ineffectiveness  is  catastrophic.  Organizations  which  manage 
for  efficiency  are  called  functional  organizations.  In  executing  his  tasks,  the  project  manag- 
er will  draw  on  expert  assistance  from  many  functional  areas  and  will  establish  lines  of 
organization  control  which  will  allow  him  to  manage  efficiently.  In  general,  he  has  two 
cardinal  rules  to  follow: 

• Do  not  do  it  yourself  - accomplish  through  the  project  organization. 
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• Organize  your  resources  to  fit  the  project  - be  prompt  and  precise  in  defining  the 
organization. 

The  project  organization  exists  within  an  organization  (which  will  normally  be  func- 
tionally organized).  In  order  to  reach  its  goals,  it  must  live  within  the  chain  of  command  of 
its  parent  organization,  and  it  must  establish  a chain  of  command  within  itself.  A chain  of 
command  is  an  organization  of  three  elements:  responsibility,  authority,  and  accountabili- 
ty. Usually  a project’s  charter  will  define  the  project  goals  without  mention  of  these  three 
elements.  Organization  instructions  may  define  project  manager  responsibilities  in  a general 
way  with  only  implications  of  assigning  authority  and  no  actual  accountability.  In  practice, 
the  project  manager  should  assume  that  he  is  fully  responsible  for  meeting  his  project  goals 
and  he  should  assume  all  the  authority  that  he  is  allowed  by  law  and  by  his  supervision  to 
meet  his  responsibilities.  Within  the  project  organization,  he  will  clearly  assign  responsibili- 
ties, delegate  appropriate  authority,  and  hold  accountable  each  responsible  individual.  The 
key  to  his  success  will  often  be  his  authority  and  his  ability  to  exercise  and  delegate  it. 
Outside  the  project,  the  manager  should  elicit  the  cooperation  of  those  who  have  authority 
above  him,  to  ensure  that  he  is  backed  up,  by  keeping  his  chain  of  command  informed 
truthfully,  concisely,  and  specifically.  Authority  is  the  power  to  make  decisions.  It  is  im- 
portant to  remember  that  small  decisions  must  be  made.  A “no  decision”  is  worse  than  a 
“wrong  decision”  because  with  the  wrong  decision  the  manager  knows  what  he  did  and  can 
correct  it;  with  no  decision,  the  situation  will  inevitably  grow  worse,  perhaps  without  any 
indication  of  the  appropriate  corrective  action.  Admiral  Nimitz  was  reputed  to  have  said, 
“the  time  for  taking  all  means  for  a ship’s  safety  is  while  you  are  still  able  to  do  so.”  Deci- 
sions are  required  to  solve  problems;  solutions  usually  result  from  perspiration  — not  inspira- 
tion. When  a manager  has  a problem,  he  has  basically  two  methods  available  to  solve  it;  the 
important  thing  is  that  the  decision  be  made. 

PROBLEM  SOLVING 

Classical  Method  Scientific  Method 

1.  What  is  the  problem?  1.  Define  the  problem 

2.  What  are  the  alternatives?  2.  State  objectives 

3.  What  is  the  best  alternative?  3.  Formulate  a hypothesis 

4.  Collect  data 

5.  Classify,  analyze,  and  interpret 
data  against  the  hypothesis 

6.  Draw  conclusions,  generalize,  re- 
state, or  develop  new  hypotheses. 

The  solution  should  be  kept  in  perspective  by  asking,  “Is  it  adequate?”  and  “Is  it  too 
elaborate?” 

In  order  to  make  decisions,  the  manager  must  be  informed.  The  manager  uses  the 
project  organization  and  procedures  to  keep  informed  of  project  activities,  usually  utilizing 
some  form  of  convenient  management  information  system.  Again,  the  solution  is  tailored 
to  the  needs.  On  small  projects,  the  project  manager  will  keep  directly  informed  about  all 
the  specifics  of  his  project.  On  large  projects,  the  project  manager  will  rely  mainly  on  re- 
ports and  plans  and  will  focus  on  exceptions  to  the  overall  plan. 
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In  order  to  obtain  up-the-Iine  cooperation  to  get  higher  order  decisions,  the  project 
manager  should  know  his  own  project  and  also  related  projects.  In  knowing  h is  own  proj- 
ect, the  manager  can  confidently  relate  accurate  information  to  his  superiors.  This  confi- 
dence and  frankness  can  play  a role  in  generating  trust  which  will  be  valuable  if  problems 
requiring  outside  assistance  arise.  Also,  the  knowledge  of  other  projects  will  assist  the  man- 
ager in  recognizing  the  parent  organization’s  perspective  and  in  establishing  a priority  to 
obtain  the  needed  decision.  Avoiding  “tunnel  mindedness”  can  be  very  helpful  when  com- 
peting for  a share  of  limited  organization  resources  especially  funding.  Avoid  “buttering 
up”  reports  to  show  only  good  news;  major  problems  cannot  be  covered  up  and  will  torpedo 
this  facade.  The  project  must  satisfy  the  parent  organization’s  goals. 

This  guide  is  offered  in  recognition  of  the  fact  that  most  project  managers  are  good 
technical  men  who  may  be  inexperienced  managers.  It  is  also  an  attempt  to  offer  practical 
methods  to  implement  the  recommendations  of  the  various  government  studies  on  reducing 
costs  (see  appendix  B),  many  of  which  are  not  addressed  by  directives  and  instructions.  It  is 
hoped  that  the  Guide  will  serve  as  a useful  navigational  tool  for  the  manager  as  ne  weaves 
his  project’s  course  through  instructions,  budgets,  specifications,  and  the  like  to  a successful 
implementation  in  the  Fleet. 
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II.  REQUIREMENTS  DEFINITION 


The  most  basic  step  in  any  successful  endeavor  is  to  thoroughly  establish 
a goal.  This  chapter  cites  the  elements  necessary  to  define  the  goal  for  an 
acquisition  project.  These  elements  determine  how  an  item  will  be  used 
and  supported  and  how  much  it  will  cost  through  its  life  cycle.  All  the 
steps  which  follow  must  point  toward  the  project  goal. 

A project  which  is  not  confined  to  basic  research  has  objectives  which  must  ulti- 
mately be  application  oriented.  The  identified  application  is  defined  in  terms  of  user 
requirements.  Usually,  these  requirements  will  be  documented  by  an  Operational  Require- 
ment (OR)  (OPNAV  INST  5000.42).  A project  not  having  an  OR  will  normally  be  part 
of  or  closely  related  to  a program  which  is  so  documented;  alternately,  a project  whose 
purpose  is  to  replace  existing  equipment  may  assume  the  same  user  requirements  as  the 
extant  equipment.  In  any  case,  certain  user-oriented  parameters  must  be  known  before 
valid  technical  requirements  can  be  generated.  The  following  requirements  elements  should 
be  identified  before  progressing  into  a conceptual  effort. 

Operational  need  - the  operational  problems  and  threats  to  be  addressed  by  the 
project  including  the  scope  and  parameters  of  the  expected  threat  environment,  deficiencies 
in  present  capabilities,  and  the  consequences  of  not  satisfying  the  need.ie.  statement  of  how 
vital  the  system  will  be  and  the  required  system  availability.  < 

Operational  concept  - a statement  of  how  the  user  is  to  use  the  new  capability  : 

to  combat  the  operational  need.  This  includes  definitive  mission  parameters;  ie.  mission 
duration,  ship-air-shore  platforms  involved,  intervals  between  missions,  the  mission  environ- 
ment, the  system  utilization  rate,  etc,  and  requirements  for  the  system  to  be  compatible 
with  other  systems.  The  number  of  systems  to  be  implemented  and  the  priorities  of 
implementation  are  included. 

Performance  goals  - the  known  performance  criteria,  operating  modes,  and  other 
system  characteristics  stated  with  minimum  acceptable  limits  and  allowable  degradation 
conditions. 

Life  support  goals 

Life  expectancy 

Special  logistic  and  training  support  considerations  and  constraints 
Nominal  goals  for  mission  and  operating  reliabilities 

Level  of  maintenance  capability  constraints  including  the  maximum  annual 
maintenance  time  at  the  organizational  level 

Cost  objectives 

Production  design-to-cost  goals  given  the  number  of  systems  to  be  acquired 
within  the  postulated  time  frame 

The  maximum  annual  maintenance  cost  at  the  organizational  level 

Initial  operational  capability  (IOC)  - the  desired  fleet  introduction  date. 

Additionally,  an  OR  will  state  whether  there  are  ongoing/related  efforts  which 
should  be  coordinated  with  the  project. 

If  any  of  the  above  requirements  elements  are  deficient,  efforts  should  be  initiated 
to  obtain  the  needed  information  prior  to,  or  as  soon  as  possible  during,  the  conceptual 
phase. 
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III.  PROGRAM  PLANNING 


This  chapter  provides  some  keystones  of  demonstrated  value  in  the  organ- 
ization, planning,  and  execution  of  a project.  The  manager  is  exhorted  to 
consider  goals,  resources,  constraining  factors,  and  risks  as  one  coherent 
entity. 

Program  planning  is  an  art  which  attempts  to  assemble  a number  of  conflicting 
elements  into  a coherent,  practical  plan  leading  to  an  acceptable  result.  These  elements 
are: 

Program  objectives 
Available  resources 
Constraining  factors 
Risks 

Thus,  the  program  plan  must  achieve  the  program  objectives  with  the  available  resources 
within  the  constraints  at  an  acceptable  risk. 

The  program  objectives  will  include,  as  a minimum,  the  satisfaction  of  the  opera- 
tional requirement.  If  the  requirements  are  not  sufficiently  well  defined,  any  undertaking 
to  satisfy  them  will  waste  valuable  resources.  Often  additional  objectives  will  be  manifest, 
such  as  “delivery  by  1979,”  “within  $30  000,”  or  “maintain  compatibility  with  what  we 
have  now.”  These  additional  objectives  may  not  be  wholly  consistent  with  the  “best” 
solution  to  the  primary  problem;  nevertheless,  they  must  be  addressed.  Sometimes  it  is 
possible  to  influence  the  statement  of  objectives.  If  so,  try  to  make  the  primary  objective 
(satisfying  the  operational  requirement)  as  clearly  defined  as  possible  and  minimize  other 
objectives.  When  secondary  objectives  are  included,  determine  how  much  flexibility  can 
be  tolerated  in  meeting  them. 

Resources  include  manpower,  talent,  facilities,  tine,  funding,  test  equipment, 
information,  and  the  like;  they  are  the  raw  materials  from  which  the  program  is  built. 

The  primary  problem  is  to  determine  what  types  and  levels  of  resources  will  be  needed. 
Manpower  and  talent  together  comprise  personnel  resources,  which  are  indispensable;  they 
are  cited  separately  to  emphasize  the  fact  that  people  and  knowledge  do  not  solve  prob- 
lems - knowledgeable  people  solve  them.  Time  is  a flexible  resource  which  is  normally 
not  a problem  unless  there  is  not  enough  (as  when  a ship  schedule  must  be  met).  Most 
of  the  other  resources  can  be  tapped  easily  enough  as  long  as  it  is  known  that  they  are 
needed.  The  most  important  resources  are  personnel  and  funding;  most  projects  succeed 
or  fail  on  the  availability  of  them.  The  typical  failure  has  an  excess  of  manpower,  a 
paucity  of  talent,  and  a lack  of  control  on  the  allocation  of  funds.  Less  is  better.  Too 
many  people  make  a project  unmanageable  and  obscure  valuable  talents  from  the  tasks  to 
which  they  should  be  applied.  If  one  could  give  the  program  objective  to  a single  engineer 
and  if  he  possessed  the  knowledge  and  talent  needed,  a “best”  product  would  result.  How- 
ever, most  projects  contain  complexities  that  an  individual  cannot  cope  with  alone,  so  a 
team  of  specialists  is  usually  assembled.  Nevertheless,  the  tendency  is  to  have  too  many 
“cooks”  on  the  project.  The  Sidewinder  missile,  Mirage  fighter,  and  the  Navy’s  first 
nuclear  propulsion  plant  were  all  designed  by  teams  of  fewer  than  40  individuals.  On  the 
other  hand,  the  C-5A  was  “designed”  by  over  1200  engineers  and  the  F-l  1 1 by  over  600. 
The  personnel  resource  requirement  should  be  based  on  the  talent  necessary  to  complete 
the  tasks  with  a minimum  amount  of  manpower.  Personnel  and  funding  are  inextricably 


related.  The  level  of  funding  required  must  be  sufficient  to  obtain  access  to  the  neces- 
sary level  of  other  resources;  the  complexities  of  funding  are  discussed  separately  later  in 
this  chapter. 

The  most  obvious  program  constraints  are  limits  on,  or  a lack  of,  resources,  par- 
ticularly funding  and  time.  Constraints  are  always  present  since  no  resource  is  infinitely 
available,  but  the  constraints  are  not  crippling  unless  they  limit  a resource  at  a level 
below  sufficiency.  If  a resource  is  constrained  below  the  level  of  necessity,  the  program 
should  be  canceled;  however,  most  constraints  simply  limit  planning  options.  Two  more 
subtle  constraining  factors  are  policy  and  politics.  Most  policies  simply  provide  guidance 
or  standardize  procedures  and  do  not  pose  planning  problems  as  long  as  they  are  known. 
Policies  which  pose  problems  which  cannot  be  tolerated  can  be  changed  or  waivered  as 
long  as  valid  justification  exists.  Request  the  change  or  waiver  from  the  originator  of  the 
policy;  if  relief  is  not  obtained,  a solution  to  the  problem  can  generally  be  found  with 
the  policymaker’s  help.  Politics  can  make  or  break  a project.  Typically,  an  otherwise 
flexible  constraint  will  become  cast  in  concrete  through  political  manipulations,  and 
adverse  decisions  will  be  made  beyond  control  of  the  project.  Every  situation  is  different, 
and  little  guidance  can  be  offered  here  except  to  know  the  strength  of  friends  and  the 
weaknesses  of  foes  and  to  try  to  let  project  friends  do  battle,  allowing  project  personnel 
to  remain  behind  the  scenes  and  seemingly  aloof  of  the  wars.  In  general,  it  is  desirable 
to  influence  constraining  factors  to  allow  a maximum  amount  of  flexibility ; this  allows 
the  project  manager  to  place  the  constraints  on  tasks  rather  than  outside  influences. 

Risks  are  always  present  in  any  program,  but  they  can  generally  be  managed  and 
reduced  to  an  acceptable  level.  Project  planning  must  provide  a framework  to  identify, 
assess,  and  act  upon  risks.  The  resources  necessary  for  these  functions  consist  of  valid 
information  and  proper  talent.  Additionally,  the  project  must  be  sufficiently  flexible  to 
accommodate  changes  which  may  be  necessary  to  manage  risks.  Risk  management  is  a 
function  mandated  to  the  project  manager  by  DoD  Directive  5000. 1 and  SECNAV  Instruc- 
tion 5000. 1 Some  techniques  and  methods  for  managing  risks  are  contained  in  chapter  XXI. 


ORGANIZATION 

The  program  plan  organizes  the  program  elements  in  a series  of  tasks  to  achieve 
the  program  objectives,  determine  resource  requirements,  and  identify  constraining  factors 
and  risks.  When  the  tasks  are  well  defined,  it  is  easier  to  make  the  estimates  required, 
particularly  estimates  of  funding  and  schedule.  As  in  any  estimating  process,  there  are 
always  latent  estimating  errors.  Assuming  reasonable  estimating  (not  chronically  optimistic 
or  pessimistic),  estimating  errors  can  be  drastically  reduced  for  the  collective  project  by 
making  increasingly  more  detailed  estimates  because  actual  estimating  errors  will  tend  to 
cancel  as  they  are  aggregated  and  errors  of  omission  will  tend  to  be  limited  by  the  thought- 
ful uncovering  of  obscure  detail  tasks. 

The  standard  method  of  organizing  this  hierarchy  of  tasks  is  the  Work  Breakdown 
Structure  (WBS).  MIL-STD-881  provides  guidance  in  constructing  WBSs.  Once  constructed, 
the  WBS  becomes  a convenient  framework  for  organizing  task  assignments,  budgeting 
resources,  assessing  risks,  estimating  life-cycle  costs,  and  a host  of  other  management  tasks. 
The  complexity  of  the  WBS  will  directly  reflect  the  complexity  of  the  program.  The  WBS 
for  simple  projects  can  easily  be  managed  by  hand;  for  complex  projects,  the  aid  of  a 
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computer  is  useful.  NOSC,  for  example,  has  a WBS  computer  program  for  this  purpose.  The 
Program  Management  Manual,  NELC  Instruction  5000.2,  calls  for  use  of  a WBS.  It  is  advisa- 
ble to  create  separate  breakdowns  using  a common  WBS  for  the  following  program  items: 

Tasks  and  resource  requirements 

Costs 

Milestones 

Risks 

Nonrecurring  and  recurring  life-cycle  costs 
Production  costs,  fixed  and  variable 

In  each  structure,  it  is  advisable  to  recognize  the  design,  fabrication,  test,  assembly,  and 
documentation  tasks  associated  with  each  level  of  complexity;  such  a breakdown  will 
improve  the  accuracy  of  estimates  made  at  the  level  even  though  it  may  not  be  either 
practical  or  desirable  to  collect  accounting  data  at  that  level. 

For  a complex  system,  a WBS  may  contain  as  many  as  seven,  eight,  or  nine 
levels.  While  the  manager  must  be  aware  that  the  detailed  levels  exist,  he  must  also  be 
careful  not  to  get  mired  and  saturated  in  details.  As  a rule  of  thumb,  no  individual 
should  attempt  to  directly  manage  more  than  four  levels;  this  figure  has  been  well  estab- 
lished by  empirical  study  across  both  government  and  industry.  A complex  project  might 
be  organized  as  illustrated  below. 

i 

Individual  Responsible  Equipment  Breakdown  Level 

1.  System 

2.  Subsystem 

3.  Sets 

4.  Groups 

5.  Units 

6.  Assemblies 

7.  Subassemblies 

8.  Modules 

9.  Parts 

Of  course,  each  project  is  different,  so  the  project  organization  must  conform  to  the  pro- 
ject peculiarities.  Organizational  boundaries,  equipment  complexities,  and  technology 
implications  may  dictate  the  organizational  structure.  In  any  case,  the  goal  is  to  create  a 
project  organization  in  which  each  task  is  assigned  to  the  most  capable  individual  possible 
giving  him  the  responsibility  and  authority  to  execute  that  task  and  providing  a clear-cut 
chain  of  command,  integrating  him  into  the  organization  as  his  task  is  integrated  into  the 
project.  The  clear  cut  chain  of  command  is  essential  to  the  accomplishment  of  the  higher- 
order  tasks  at  the  system/subsystem  level  and  to  the  maintenance  of  accountability  among 
those  working  on  the  project. 

Obviously,  many  tasks  are  contingent  upon  the  completion  of  other  tasks.  The 
relationships  between  tasks  are  important  to  two  essential  management  functions:  tracking 
progress  and  risk  management.  When  estimating  the  time  required  to  complete  a task,  it 
is  important  to  consider  the  impact  on  other  tasks  if  schedule  must  be  slipped  or  cannot 
be  made  at  all.  If  this  effort  is  included  in  the  initial  planning,  realistic  alternative  plans 
can  be  developed  and  appropriate  allowances  can  be  made  for  schedule  slippages.  A 
common  error  in  milestone  planning  is  to  assume  that  each  task  will  be  completed  in  an 
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Deputies  (one  per  group) 

Lead  Engineers 


m-3 


expected  (most  likely)  time.  In  practice,  each  task  will  be  completed  early  or  late  with 
the  average  of  all  tasks  at  the  expected  time.  But  early  completions  of  a subtask  frequently 
do  not  contribute  to  an  early  completion  of  the  larger  task,  whereas  a late  completion  of 
a subtask  may  make  late  completion  of  the  larger  task  inevitable.  Studies  of  large  numbers 
of  projects  in  the  government  and  industry  show  an  average  schedule  slippage  of  25% 
beyond  the  expected  end  date.  Careful  milestone  planning  will  compensate  for  inherent 
slippages  so  that  the  end  objective  is  reached  when  promised. 


MANAGEMENT  INFORMATION  SYSTEMS 

Project  management  must  be  cognizant  of  the  progress  and  problems  of  the  tasks 
in  order  to  correct  problems  early  and  to  steer  the  project  to  a successful  completion. 

It  is  impossible  to  wholly  replace  personal  contact  between  the  manager  and  the  perfor- 
mers; however,  very  large  and  complex  projects  make  adequate  personal  contact  difficult 
to  achieve.  Literally  hundreds  of  management  information  systems  have  been  evolved 
to  assist  managers  in  keeping  track  of  their  projects.  To  be  useful,  a management  infor- 
mation system  must  have  the  following  characteristics: 

The  manager  must  be  familiar  with  it. 

Reporting  personnel  must  understand  it. 

It  must  track  all  the  elements  critical  to  the  particular  project  (and  preferably  no 

others)  (note:  these  vary  from  project  to  project). 

It  must  be  timely  and  accurate. 

It  must  be  easy  to  implement. 

Management  information  systems  may  run  the  gamut  from  informal  memos  to  highly 
structured  computerized  systems.  From  the  characteristics  above,  it  can  be  inferred  that 
the  system  is  nothing  more  than  a method  of  communicating  essential  information.  Often 
the  system  also  provides  a formal  record  of  progress  among  organizations.  Many  of  the 
systems  are  expensive  to  implement  because  they  require  extensive  changes  to  the  reporting 
organization’s  way  of  doing  business.  Wherever  possible,  the  manager  and  the  reporting 
organizations  should  determine  a system  by  mutual  agreement.  Also,  ask  only  for  infor- 
mation which  can  be  used:  the  purpose  of  an  information  system  is  defeated  by  extraneous 
data  which  can  obscure  real  problem  areas. 

It  is  far  better  to  limit  the  size  of  a project  so  that  personal  contact  can  be  used 
for  management  information  than  to  have  to  implement  a management  information  sys- 
tem in  the  first  place. 

FUNDING 

No  acquisition  program  can  even  begin  without  funding.  No  matter  how  badly 
the  Fleet  wants  or  needs  the  product  of  a good  idea,  the  product  cannot  reach  the  Fleet 
unless  a sponsor  is  found  for  the  acquisition  program.  Within  the  vast  acquisition  com- 
munity bureaucracy,  there  are  a number  of  pots  of  discretionary  funds.  To  tap  these 
funds,  the  project  must  appeal  to  the  prospective  sponsor  and  must  be  something  for 
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which  he  can  gain  support  through  his  normal  budget  channels.  Under  these  conditions, 
it  is  possible  to  get  some  funding  to  get  started.  However,  discretionary  funds  are  only  a 
mechanism  to  get  a small  amount  of  funding  (usually  less  than  $100k).  The  full  project 
funding  must  be  obtained  through  the  budget  cycle;  the  discretionary  funds  should  nor- 
mally be  applied  toward  project  planning  activities  leading  to  budgetary  estimates.  The 
cycle  of  activities  which  lead  to  the  formation  of  the  DoD  budget  and  (it  is  hoped)  to 
the  approval  of  project  funds  is  a long  one  fraught  with  pitfalls;  the  project  manager 
should  be  aware  of  these  activities  and  plan  his  tasks  to  deliver  the  right  information  to 
the  right  people  at  the  right  time.  The  budget  cycle  is  formalized  in  the  Manning,  Pro- 
gramming, and  Budgeting  System  (PPBS).  Virtually  all  planning  in  the  bureaucratic 
structure  above  the  project  manager  is  oriented  toward  the  budget.  Programming  is  de- 
ciding which  budget  plans  will  be  put  into  the  budget  submission.  Budgeting  is  the  dis- 
pensing of  approved  funds  and  reprogramming  those  funds  as  necessary  to  cover  budget 
overruns.  Each  level  in  the  chain  of  command  has  some  authority  in  the  review  and 
reprogramming  of  funds;  project  funding  is  susceptible  to  alterations  at  each  level.  The 
higher  review  levels  receive  only  abbreviated  information  about  the  budget  elements,  so 
changes  at  high  levels  are  frequently  somewhat  arbitrary  and  not  within  the  influence  of 
the  project  manager.  On  the  other  hand,  the  levels  near  the  sponsor  can  be  influenced. 
The  project  manager  can  greatly  enhance  the  project  funding  prospects  by  developing 
strong  project  justifications,  by  marshalling  user  support,  and  (most  important)  by  select- 
ing a strong  sponsor.  Equally  important  is  the  timing  of  proposals  and  justifications; 
figure  III- 1 shows  the  PPBS  cycle  with  the  points  where  action  may  be  indicated.  For  a 
more  detailed  explanation  of  the  PPBS,  see  DoD  Instruction  7045.7,  SECNAV  Instruction 
5000. 1 6D,  and  the  Navy  Programming  Manual ; an  excellent  overview  is  presented  in 
ARINC  Publication  1313-01-1-1447  of  15  October  1975  (section  4.3). 

When  making  funding  estimates,  characterize  the  estimates  in  accordance  with 
OPNAV  Instruction  7040.5;  for  convenience  the  appropriate  categories  are  listed  in 
table  IIM. 

When  formulating  justification,  emphasize  the  project  features  which  are  most 
relevant  to  the  interests  of  the  reviewers  and  describe  them  in  terms  appropriate  for  the 
budget  category  of  the  funds  being  made  available  to  the  project.  The  budget  categories 
are  described  in  table  II1-2. 

Important.  The  budget  cycle  is  at  least  2 years  long,  especially  when  large  dollar 
amounts  for  procurements  are  involved;  therefore,  thorough  and  accurate  advanced  plan- 
ning is  essential  to  ensure  that  funding  is  available  to  support  project  activities. 


PROJECT  DOCUMENTATION 

The  number  and  variety  of  documents  required  simply  because  a project  exists  are 
enough  to  stagger  the  imagination.  Unless  the  project  manager  is  prepared,  the  documen- 
tation requirements  will  inundate  him.  Many  of  the  documents  are  necessary  to  obtain 
funding  or  to  gain  various  forms  of  approval.  The  documents  necessary  to  establish  the 
project  at  NOSC,  for  example,  are  covered  by  the  Program  Management  Manual,  NELC  In- 
struction 5000. 2 A.  Documents  supporting  procurements  are  cited  by  the  NELC  Procure- 
ment Procedures  Manual.  The  documentation  which  may  be  required  in  the  course  of  the 
project  in  conjunction  with  the  tasks  is  discussed  in  the  Documentation  section  of  this  guide 
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(see  chapter  20).  Despite  all  these  diverse  papers,  there  are  still  documents  which  will  be 
required  in  one  form  or  another  during  the  project.  The  sponsor  may  author  some  of 
these  documents  or  may  task  the  project  to  generate  them.  Table  III-3  summarizes  the 
documents,  their  purpose,  and  the  guidance  to  their  preparation  where  available. 

Usually,  there  is  some  flexibility  in  the  preparation  of  the  documents.  Often  it 
is  possible  to  survey  the  various  required  documents  and  to  write  a few  paragraphs  which, 
with  some  cutting  and  pasting,  can  satisfy  all  the  requirements.  At  the  same  time,  there 
is  no  point  in  writing  a series  of  paragraphs  filled  with  pap.  Try  to  make  the  documents 
meaningful,  concise,  and  accurate;  then  the  few  times  someone  actually  uses  them,  they 
will  work  for  the  project. 


CATEGORY 

A 

B 


C 


D 


F 


TABLE  lll-l . COST  ESTIMATE  CATEGORIES. 

DESCRIPTION 

Contract  for  item(s)  under  consideration  in  existence.  Cost  estimate 
based  on,  included  in,  or  modifying  existing  contract.  In  operating 
accounts,  estimate  based  on  firm  pay  rates  or  firm  operating  pro- 
gram costs  with  requirements  defined  in  detail. 

Request  for  proposal  (RFP)  for  item(s)  under  consideration  in  hand. 
Cost  data  based  on  Navy’s  evaluation  of  the  RFP.  In  operating 
accounts,  estimate  based  on  approved  budgeted  pay  rates  or  opera- 
ting program  costs  with  average  requirement  accurately  defined. 

Lowest  budget  quality  estimate.  Based  on  engineering  analyses  of 
detailed  characteristics  of  the  item(s)  under  consideration.  In  oper- 
ating accounts,  estimate  based  on  proposed  budget  pay  rates  or 
operating  program  costs  with  requirements  defined  in  total. 

Estimate  based  on  technical  feasibility  studies  and/or  extrapolation 
from  higher  quality  estimates  for  similar  item(s).  In  operating 
accounts,  pay  rates  or  operating  program  costs  should  be  at  least 
category  C but  operating  requirements  are  defined  only  to  + or  -10% 
order  of  magnitude. 

Ballpark  estimate.  Proposal  required  further  definition  or  further 
study  to  refine  costs  where  a higher  category,  save  for  one  or  two 
requirements  thereof,  would  be  justified;  this  should  be  spelled  out. 
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TABLE  III-2.  NAVY  BUDGET  CATEGORIES. 

Research,  Development,  Test  and  Evaluation,  Navy  (RDT&EN) 

6.1  Basic  Research 

6.2  Exploratory  Development/Applied  Research 

6.3  Advanced  Development 
6.3a  Technology  originated 

6.3b  Operational  Requirement  originated 

6.4  Engineering  Development 

6.5  Systems  Engineering  Support/Operations  Analysis 

6.6  Operational  System  Development  and  Unallocated 
RDT&E  (to  cover  emergent  requirements) 

Military  Personnel,  Navy  (MPN) 

Operation  and  Maintenance,  Navy  (OMN) 

Procurement  of  Aircraft  and  Missiles,  Navy  (PAMN) 

Shipbuilding  and  Conversion,  Navy  (SCN) 

Other  Procurement,  Navy  (OPN) 

Military  Construction,  Navy  (MCN) 

“Other”  Navy  (Naval  Reserve  and  Marine  Corps) 


TABLE  III-3.  PROJECT  DOCUMENTS. 


Category/Document 

Requirements 

Operational  Requirement  (OR) 
Development  Proposal  (DP) 

Navy  Decision  Coordinating  Paper  (NDCP) 
Science  & Technology  (S&T)  Objectives 
Advanced  System  Concept  (ASC) 


Purpose 


Project  objectives 
Project  approach 
Development  concept 
General  technical  goals 
Describes  a technical  solu- 
tion to  an  operational 
problem 


Reference 


OPNAVINST  5000.42A 
OPNAVINST  5000.42A 
OPNAVINST  5000.46 
OPNAVINST  5000.42A 
NAVMATINST  5000.22 
NAVMATINST  39 10. 10C 


Program  Planning 
Program  Management  Plan  (PMP) 


Test  and  Evaluation  Master  Plan  (TEMP) 
Integrated  Logistic  Support  Plan  (ILSP) 


Identifies  project  objec- 
tives, resources,  con- 
straints, risks,  and  man- 
agement approach 
Identifies  T&E  issues, 
objectives,  and  resources 
Documents  support  con- 
cepts, objectives,  and 
constraints;  includes 
plans  and  shows  relation- 
ships between  reliability, 
maintainability,  human 
engineering,  safety,  per- 
sonnel & training,  provi- 
sioning, transportability, 
etc, tasks 


NAVMATINST  5200.1  IB 


OPNAVINST  3960. 10 


SECNAVINST  4000.29A 


OPNAVINST  4I00.3A 


NAVMATINST  4000.20B 
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TABLE  III-3.  (Continued) 


Category/Dovqpient 

Purpose 

Reference 

i v 

Program  Planning  (cont) 

Configuration  Management  Plan  (CMP) 

Management  approach  to 
maintain  configuration 
control 

NAVMATINST  4 130.1  A 

Data  Management  Plan 

Procedures  for  processing 
and  distributing  project 
data 

NAVMATINST  4000.1  SA 

Security  Plan 

Project  security  procedures 

OPNAVINST  5000.1  E 
DOD1  5200.1  R 

Proposed  Military  Improvement 

Initiates  installation 
planning 

OPNAVINST  4720.20 

Procurement  Plans 

Advanced  Procurement  Plan  (APP) 

Details  planned  use  of  con- 
tractual resources 

NAVMAT  P4202  (Navy 
Procurement  Directives) 
NPD-1-2100 

Request  for  Authority  to  Negotiate  (RAN) 

For  negotiations  to  procure 
experimental,  develop- 
mental, or  research  work; 
must  be  signed  by  an 
authorized  official 

NPD-3-303 

Justification  for  Authority  to  Negotiate 
(JAN) 

May  be  used  instead  of 
RAN 

NAVMATINST  3900.3B 

Miscellaneous 

Research  & Technology  Work  Unit  Sum- 
mary (DD1498) 

Establishes  a record  of  the 
existence  of  the  project 

NAVMATINST  3910.15A 

Vahie  Engineering  Plan 

Establishes  VE  procedures 

SECNAVINST  4858.2B, 
NAVMATINST  4858.8A, 
MIL-V-38352. 

Quality  Assurance  Plan 

Establishes  project  proce- 
dures for  quality  review, 
verification,  and 
certification/acceptance 

NAVMAT  4855.1 , MIL-Q- 
9858A,  or  MIL-1-45208. 

Standardization  Plan 

Determines  the  level(s)  of 
standardization  to  be  im- 
plemented and  parts  se- 
lection criteria 

MIL-STD-680 
NAVMATINST  41 20.97 A 

■ 


‘impact  point,  action  must  be  taken  prior  to  date  indicated 

© MAJOR  PROGRAMS  ONLY  (>$50M) 

© LESS  THAN  MAJOR  PROGRAMS  AND  MAJOR  PROGRAMS  ONLY  <>$20M) 

© SMALL  PROGRAMS  AND  LARGER  ONLY  (>$500k) 

© MINOR  PROGRAMS  AND  SMALL  TASKS  «$500k) 

© SMALL  TASKS  ONLY  «$30k) 

Figure  III- 1 . Planning,  Programming,  and  Budgeting  System  (PPBS)  cycle. 
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PROGRAM  TAILORING 


If  one  could  give  the  program  objective  to  a single  engineer  and  if  he  possessed  the 
knowledge  and  talent  needed,  a “best”  product  would  result.  This  statement  is  worth  re- 
peating in  order  to  emphasize  the  necessity  of  keeping  a project  manageable.  On  the  other 
hand,  most  projects  vastly  exceed  any  single  engineer’s  knowledge,  talent,  or  capability  to 
execute  the  needed  tasks  within  a specified  time,  so  the  project  team  approach  becomes  the 
practical  solution. 

The  concept  of  doing  just  enough  to  do  a thorough  and  adequate  job  is  called  tailor- 
ing. Most  of  this  guide  is  dedicated  to  tailoring  concepts.  For  instance,  chapter  XX  includes 
procedures  for  tailoring  documentation  requirements.  The  “cover  your  anatomy”  approach 
to  equipment  acquisition  is  extremely  expensive  and  wasteful  of  resources,  and  often  leads 
to  project  failure  by  emphasizing  nonproductive  tasks.  The  techniques  of  tailoring  consti- 
tute a balancing  act  between  insufficiency  and  overkill.  The  essence  of  all  the  advice  in  the 
chapters  which  follow  is,  “Require  exactly  that  which  is  needed.” 

The  problem  of  program  tailoring  is  that  an  almost  infinite  amount  of  knowledge  is 
required  in  order  to  absolutely  define  what  is  needed.  Gathering  this  knowledge  expends 
project  resources.  Eventually  a point  of  diminishing  returns  results  wherein  the  new  knowl- 
edge gained  requires  more  resource  expenditure  than  what  can  be  saved  in  the  acquisition  of 
equipment.  At  some  point  prior  to  the  point  of  diminishing  returns,  there  will  be  sufficient 
confidence  in  the  information  at  hand  that  decisions  can  be  made  with  an  acceptably  low 
risk.  What  constitutes  an  acceptable  risk  depends  on  the  consequences  of  the  decision, 
which  in  turn  depend  on  the  magnitude  of  the  project.  A very  small  project  can  tolerate 
higher  risks  because  a complete  project  failure  constitutes  a small  loss.  A very  large  project 
demands  low-risk  decisions  because  even  a partial  failure  constitutes  a very  large  loss. 

The  information  level  needed  for  very  small  efforts  (the  one  or  two-man  efforts  on 
the  order  of  SI 00k  or  less)  can  be  satisfied  by  educated  guesses  in  most, cases.  The  individ- 
ual(s)  executing  the  tasks  should  be  thoroughly  experienced  in  the  critical  tasks  to  be  per- 
formed, such  as  design  or  installation  planning  or  testing.  On  less  critical  tasks,  appropriate 
expert  advice  should  be  located  on  a free  or  part-time  basis.  The  task  leader  should  be 
responsible  for  researching  noncritical  issues  to  his  satisfaction.  The  success  of  the  project 
will  hinge  on  the  selection  of  task  personnel  and  on  the  proper  determination  of  critical 
issues. 

At  the  extreme  of  the  very  large  project  (major  program  as  defined  in  DoD  Directive 
5000.1),  supporting  research  and  test  projects  should  be  utilized  to  validate  information  as 
well  as  gather  it.  Full-time  expert  assistance  will  be  utilized  for  a large  range  of  issues.  The 
size  of  the  program  will  limit  the  number  of  issues  which  can  be  treated  as  noncritical. 

To  illustrate  program  tailoring,  consider  the  issue  of  reliability  as  it  might  be  treated 
by  projects  of  various  sizes.  In  a very  small  project,  reliability  can  probably  be  ignored  as  an 
independent  issue;  good  design  practices  can  be  depended  on  to  yield  sufficient  reliability. 
The  small  project  might  require  a reliability  prediction  for  support  planning  purposes,  but  a 
cursory  prediction  using  MIL-HDBK-217  can  suffice.  An  intermediate  project  would  re- 
quire a more  detailed  prediction  and  would  probably  require  a failure  modes  and  effects 
analysis  (FMEA);  a reliability  specialist  would  be  employed  to  assist  in  these  tasks.  Also, 
intermediate  projects  may  use  reliability  testing  as  a portion  of  qualification  tests.  Large 
and  very  large  projects  may  employ  a full-time  reliability  specialist  to  directly  influence  the 
equipment  design  and  to  perform  other  reliability  tasks;  reliability  testing  should  play  a 
significant  role  in  the  reliability  program  as  a forma!  reliability  growth  technique  may  be 
utilized.  These  brief  descriptions  show  how  a given  issue  gains  importance  with  project  size 
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CONFIDENCE  LEVEL 


■ 


and  management  response  grows  accordingly.  In  each  case,  it  is  assumed  that  reliability  is 
not  a critical  issue  in  the  operational  requirement;  if  it  were,  a more  intensive  effort  would 
be  made  for  each  project  size. 

Table  111-4  describes  the  levels  of  information  which  give  various  degrees  of  confi- 
dence to  the  decision-making  process.  Refer  to  chapter  XIX  for  information  regarding  the 
test  and  evaluation  procedures  which  arc  essential  to  high-confidence  decisions.  Figures 
1 1 1-2  and  1 1 1-3  are  provided  to  help  conceptualize  the  relationship  between  project  size  and 
the  acceptable  levels  of  information  for  project  decisions. 

Require  what  is  needed,  reject  the  unnecessary. 

TABLE  1 1 1-4.  LEVELS  OE  INFORMATION  FOR  DECISION  MAKING. 


0.  Noneducated  guess 

1 . Educated  guess  by  a nonexpert 

2.  Educated  guess  by  an  expert 

3.  Expert  advice 

4.  Research  and  analysis 

5.  Analysis  and  simulation 

6.  Diagnosis  of  prior  experience 

7.  Partial  testing  in  use  (such  as  a Fleet  Research  Investigation  or 
Development  Assist) 

8.  Full-scale  testing  in  use  (such  as  an  Operational  Assist  or  OPEVAL) 


CRITICAL  ISSUES 


guesses 

information  developed 
from  experience 


validated  information 


LEVEL  OF  INFORMATION 


2 3 4 5 6 

LEVEL  OF  INFORMATION 


Figure  II 1-2.  Level  of  information  (from  table  I1I-4)  vs  confidence  and  effort  to  obtain. 


ACQUISITION  PROJECT  SIZE 


PRODUCTION  $ 


EGORY 

RDT&E  $ 

1 

> 50M 

> 200M 

(VERY  LARGE) 

II 

> 20M 

> 50M 

(LARGE) 

III 

>5M 

> 20M  I 

| (INTERMEDIATE) 

IV  A 

> 1 M 

> 5M  j 

IV  B 

> 300k 

> 1M  j 

| (SMALL) 

IV  C 

> 100k 

> 300k  j 

IV  D 

EVERYTHING 

ELSE 

(VERY  SMALL) 

LOW  PRIORITY  HIGH  PRIORITY 

RANGE  OF  PROJECT  ISSUES 
Figure  III-3.  Project  size  vs  range  of  project  issues. 
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IV.  CONCEPTUAL  PHASE 


IV.  CONCEPTUAL  PHASE 


Decisions  made  during  the  conceptual  phase  dedicate  a significant  portion 
(typically  70%)  of  a system’s  total  life-cycle  costs;  sometimes  costs  can  be 
affected  by  as  much  as  two  orders  of  magnitude.  In  order  to  achieve  the 
most  overall  performance  at  the  lowest  cost,  the  conceptual  phase  must 
consider  the  total  user  requirements  while  completing  the  normal  tasks  of 
the  project  phase.  This  chapter  discusses  the  tasks  to  be  performed  and 
ways  to  integrate  the  total  requirement  into  them.  The  management  of 
technical  risk  is  included  as  a primary  task. 


INTRODUCTION 

The  conceptual  phase  of  any  program  or  project  is  critical  to  its  ultimate  success 
or  failure.  In  general,  this  phase  is  placed  in  jeopardy  by  the  lack  of  resources  made 
available  to  resolve  the  very  large  uncertainties  bearing  upon  the  feasibility  of  applying 
technologies  to  perform  the  tasks  needed  to  satisfy  the  stated  requirement.  The  problem 
thus  posed  is  further  exacerbated  by  the  ability  to  defer  unresolved  questions  into  later 
program  phases.  On  the  other  hand,  it  would  require  an  inordinate  amount  of  time  and 
expenditure  of  resources  to  completely  resolve  all  pertinent  issues.  Some  amount  of  risk 
must  be  assumed  in  order  to  satisfy  the  stated  requirements  expeditiously.  Therefore, 
the  tasks  of  the  conceptual  phase  are  twofold: 

To  formulate  feasible  approaches  to  the  problem  solution 

To  assess  and  manage  the  risks  associated  with  each 

These  tasks  are  normally  performed  in  two  closely  associated  steps  known  as  concept 
formulation  and  technical  approach  development.  The  first  step  delineates  one  or  more 
promising  concepts;  the  second  integrates  into  the  overall  program  plan,  assesses  risks,  and 
provides  proof  of  feasibility  (or  infeasibility)  for  each  concept.  See  figures  IV- 1 and  -2. 

A most  important  element  in  concept  formulation  is  ensuring  that  as  complete  a 
set  of  factors  as  possible  is  used  to  identify  promising  approaches;  these  factors  include, 
but  are  not  limited  to,  the  following: 

• Functions,  modes,  inputs,  and  outputs  dictated  by  the  requirements 

• Mission  constraints,  such  as  reaction  time,  probability  of  detection,  and  proba- 
bility of  error,  preferably  embodied  in  an  effectiveness  model 

• Support  constraints,  including  minimum  acceptable  values,  for  reliability,  main- 
tainability, availability;  goals  for  operability  and  transportability;  and  safety 
requirements 

• Environmental  constraints  tailored  to  the  worst-case  conditions  under  which  the 
equipments  are  expected  to  operate  (not  necessarily  full  MIL-SPEC) 

• Cost  constraints  in  both  acquisition  and  support  of  the  equipments,  assessed 
through  an  LCC  model 

• Producibility  constraints  arising  from  the  quantities  required  in  the  near  term 
and  far  term 
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DETERMINE  METHOD 
OF  FEASIBILITY 
DEMONSTRATION: 
ANALYSIS,  SIMULATION, 
EXPLORATORY 
DEVELOPMENT 


CHECK  COMPATIBILITY 
OF  RESOURCE  REQUIREMENTS 
WITH  PROGRAM  CONSTRAINTS 


COMPATIBLE? 


PROGRAM 

CONSTRAINTS 

(SEE  CH  3) 


CONSTRAINTS 
WAI VERABLE 

. ? / 


reevaluate 

RESOURCE 

_fc^/^RESOURCESSs 

^YES 

—J. 

PROVE 

1 

— OBTAIN 

REQUIREMENTS 

FNvADEQUATE? 

FEASIBILITY 

WAIVER 

/CANCEL  T 

IconceptJ 


(DEFICIENCIES] 


r 1 NO  / 

CANCEL  f*  CONTINUE? 


DEFICIENCIES  & k^^PR^?"^^ 
REQUIREMEN7  S I 'N^PROVED 


WRITE  SYSTEM  > 
SPECIFICATION 
TO  VALIDATION 
.PHASE  J 


Figure  IV-2.  Technical  approach  (for  each  possible  concept). 
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Ultimately,  the  system  must  be  capable,  supportable,  and  affordable.  The  impor- 
tance of  a well  defined  operational  requirement  as  a baseline  to  this  effort  cannot  be  over- 
stressed. The  methods  of  defining  requirements  depend  on  the  type  of  requirement.  In 
general,  operational  requirements  will  fall  into  one  of  the  following  categories: 

1 . Replacement  of  an  existing  capability  or  group  of  capabilities 

2.  Replacement  of  existing  capabilities  plus  additional  features  which  enhance 
those  capabilities 

3.  Replacement  of  existing  capabilities  plus  new  capabilities 

4.  Supplanting  of  existing  capabilities  by  new  capabilities 

5.  Entirely  new  capabilities 

A capability  is  an  ability  to  accomplish  a sub-mission  (ie,  to  detect,  to  communicate,  to 
track);  the  sum  of  capabilities  constitutes  an  ability  to  perform  operational  missions  (ie, 
ASW,  AAW,  gunfire  support,  convoy  escort).  Requirements  to  replace  an  existing  capa- 
bility (categories  1-3)  can  usually  be  stated  as  deficiencies  to  be  corrected  or  improved 
upon  while  assuming  nondeficient  characteristics  as  extant.  Category  1 requirements  almost 
always  arise  from  obsolescent  equipments  which  are,  or  are  becoming,  difficult  to  maintain, 
hard  to  support,  unreliable  compared  to  what  can  now  be  attained,  and  so  forth;  detailed 
information  on  the  support  characteristics  and  support  parameters  (MTBF,  MTBCM, 

MTTR.  An,  MDT)  should  be  obtained  on  the  existing  equipments  and  used  to  establish 
minimum  acceptable  parameters  and  design  goals  for  the  current  project.  Category  2 
requirements  generally  involve  performance  deficiencies  but  may  also  contain  support 
deficiencies;  the  required  performance  characteristics  may  usually  be  generated  through 
threat  analysis  and  operations  analysis,  if  they  are  not  explicitly  stated  in  the  requirements 
document  Category  3 requirements  usually  arise  from  expanding  missions  or  where  it  has 
become  desirable  to  automate  previously  manual  functions;  category  3 performance  char- 
acteristics will  tend  to  be  more  operationally  oriented  than  those  of  category  2,  but  are 
similar  in  the  steps  needed  for  their  derivation.  All  requirements  based  in  existing  capa- 
oilities  (categories  1-4)  should  depend  on  a detailed  knowledge  of  those  capabilities  and 
their  deficiencies  in  performance  and  supportability  to  derive  the  goals  for  the  new 
equipments. 

Requirements  based  in  new  capabilities  to  some  extent  (categories  3-5)  must  have 
definitive  characteristics  stated;  if  not  supplied  in  the  requirements  document,  threat 
analysis  and  operations  analysis  will  be  necessary  in  some  degree.  However,  the  best 
method  (in  terms  of  the  accuracy  of  information  obtained)  for  determining  the  required 
characteristics  is  a Fleet  Investigation  (see  OPNAV1NST  3960.10  series).  The  Fleet  inves- 
tigation should  be  instrumented  to  measure  all  parameters  pertinent  to  the  particular 
threat  scenario,  so  that  effective  methods  of  simulation  and  accurate  test  plans  may  be 
generated  to  support  the  proof  of  feasibility  phase. 

As  a guiding  policy  replacement  capabilities  should  be  at  least  as  good  in  perfor- 
mance supportability  as  replaced  capabilities  at  less  total  life  cost;  or  should  supply  im- 
proved performance/supportability  at  the  same  cost;  or  both.  New  capabilities  should  be 
incorporated  within  well  defined  bounds  for  acquisition  cost,  support  costs,  and  logistics 
requirements. 


Ultimately,  the  operational  requirements  must  be  transformed  into  technical 
(specification)  requirements.  The  following  sections  discuss  the  formulation  of  the 
various  technical  parameters;  however,  a guiding  force  in  the  ensuing  tradeoffs  must  be 
the  criticality  of  the  system  or  equipments  to  the  operating  organization  (ship,  airwing, 
etc)  which  it  services.  The  following  definitions  are  provided: 

Vital  Equipments  which  are  essential  to  the  primary  mission(s)  of 

the  organization  or  for  damage  control  or  for  personnel  safety 

Semivital  Equipments  which  are  essential  to  secondary  missions  or  which 
are  supportive  to  the  primary  missions,  damage  control,  or 
personnel  safety 

Nonvital  All  other  equipments 

Note  that  a vital  system  (such  as  uhf  shipboard  communications)  might  be  made  up  of 
semivital  equipments  by  virtue  of  the  inherent  system  redundancy.  Conversely,  an  equip- 
ment which  is  common  to  a number  of  semivital  systems  might  be  considered  vital  if  no 
sufficient  backup  is  provided.  (No  equipment  is  currently  known  to  fit  this  circumstance, 
but  the  situation  is  described  as  one  to  be  avoided.)  These  categories  are  useful  in  estab- 
lishing: 

The  level  of  risk  which  can  be  acceptably  assumed  without  terminating  the  pro- 
gram (see  table  IV-  i ) 

The  acceptable  limits  for  support  parameters,  especially  mean  downtime  and 
maximum  downtime 

The  priorities  and  proportional  shares  of  funding  and  manpower  resources  which 
can  reasonably  be  allocated  to  the  equipment  in  development,  procurement,  and 
support 

On  the  last  point,  it  is  obvious  that  the  cost-benefit  payoffs  must  be  significantly 
higher  for  a nonvital  or  semivital  equipment  to  justify  equal  consideration  with  a more 
vital  equipment  competing  for  the  same  resources.  The  determination  of  system  criticality 
is  normally  stated  in  the  requirements  documentation,  although  sometimes  implicitly, 
and  must  be  considered  in  the  program  planning  (see  chapter  III),  as  well  as  in  the  deter- 
mination of  support  parameters  (see  SUPPORT  PARAMETERS  in  this  chapter). 

The  above  considerations  require  a great  deal  of  information  in  order  to  make 
intelligent  decisions  early  in  the  conceptual  phase.  In  large  measure,  the  manager's  ability 
to  control  risks  and  to  achieve  ultimate  project  success  rests  in  his  knowledge  of  what 
uncertainties  exist  and  his  utilization  of  available  information  to  make  correct  determina- 
tions. The  next  section  deals  with  prospective  sources  of  information. 
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TABLE  IV-1.  INITIAL  RISK  ASSESSMENT. 


System  Character 


Risk  Assessment 


Action 


The  major  items  in  the  system  have  low 

been  previously  developed  and  used 
together.  (Development  of  noncritical 
ancillary  items  is  permitted.)  (Minor 
modifications  are  allowed.) 


Choose  a Preferred  Approach  which 
seems  most  promising  to  meet  project 
objectives;  several  approaches  may  be 
considered  preferred  if  no  clear-cut 
advantages  exist  for  one  over  the 
others. 


The  major  items  in  the  system  have  moderate 

been  previously  developed  but  have 
not  been  used  together  before. 


Choose  a Primary  Approach  (identical 
to  Preferred  Approach  under  low-risk 
conditions)  end  at  least  one  Secondary 
Approach  differing  from  the  Primary 
Approach  in  the  area  of  highest  risk 
(most  technical  uncertainty). 


One  or  more  of  the  major  items  in  the  high 
system  require  development. 


Choose  Parallel  Approaches  for  each 
item  requiring  development.  At 
least  one  of  the  approaches  should  be 
as  conservative  as  possible  (ie,  involving 
minimal  uncertainties). 


GATHERING  INFORMATION 


GENERAL 

Before  putting  pencil  to  drawing  board,  the  engineer  will  want  to  obtain  as  much 
information  as  possible  about  the  technology  available  for  application  to  his  design  prob- 
lem. This  section  is  intended  to  serve  as  a review  of  sources  of  information  that  the 
engineer  can  use  to  determine  ongoing  and  past  technology  that  applies  to  his  forthcoming 
system/equipment  formulation.  The  material  obtained  from  this  search  will  allow  for 
valid  design  decisions  as  discussed  in  the  next  sections,  decisions  that,  it  is  hoped,  will 
result  in  an  optima]  product  in  terms  of  cost  and  effectiveness. 

Perhaps  the  most  satisfying  and  assuring  way  to  determine  what  is  really  going  on 
is  to  have  a mouth-to-mouth  confrontation  with  other  engineers  currently  or  recently 
engaged  in  the  type  of  work  of  interest.  For  such  a dialog  to  be  of  greatest  benefit,  how- 
ever, the  inquirer  should  have  previously  obtained  as  much  background  information  as 
possible  and  some  knowledge  about  the  person  to  whom  he  is  talking  and  that  person's 
work.  For  NOSC  engineers,  the  Research  Library  has  practically  all  the  sources  of  infor- 
mation needed  to  prepare  for  meaningful  discussion  and  later  design  decisions.  As  a 
general  rule,  every  information  gathering  effort  should  begin  with  a request  for  assistance 
from  the  Public  and  Information  Services  Librarian  to  save  time  and  avoid  trouble. 

The  remainder  of  this  section  covers  the  information  services  of  research  libraries, 
two  outside  sources  of  information,  design  engineering  department  services,  and  the  Military 
Parts  Control  Advisory  Group. 
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RESEARCH  LIBRARY  SERVICES 


Library  services  are  commonly  available  to  personnel  seeking  information  on  availa- 
ble technology  and  related  industry  and  government  organizations  and  personnel.  The 
following,  all  available  at  NOSC,  are  typical. 

Defense  Documentation  Center  (DDC) 

Visual  Search  Microfilm  File  (VSMF) 

Lockheed  Information  Retrieval  Service  DIALOG 

Systems  Development  Corporation  ORBIT  II 

Western  Research  Applications  Center  DATACON 

Army  ECOM  Joint  Electronics  Type  Designations 

Miscellaneous  Publications 

DEFENSE  DOCUMENTATION  CENTER  (DDC) 

The  DDC  is  the  Department  of  Defense  central  facility  for  RDT&E  information. 
One  function  of  the  DDC  is  to  acquire,  store,  announce,  retrieve,  and  provide  secondary 
distribution  of  scientific  and  technical  documents  directly  to  registered  users,  including 
NOSC  and  its  personnel.  Quick  service  is  provided  by  the  Defense  RDT&E  On-Line 
System  of  the  DDC  which  connects  the  Library  directly  with  the  DDC  RDT&E  data 
banks/Univac  1108  by  means  of  a CRT  display  keyboard  in  the  library.  Information 
services  available  from  DDC  are:  Announcement;  Technical  Report  Copy;  Bibliography; 
Selective  Dissemination  of  Information;  DD  1498  Work  Unit  Information;  DD  1634  data 
bank;  IR&D  data  bank;  and  Referral. 

Announcement  Services  comprise  listings  of  additions  to  DDC’s  technical  report 
collection  as  announced  in  the  Commerce  Department’s  Government  Research  Announce- 
ments and,  for  classified  and  limited-distribution  documents,  DDC’s  Technical  Abstract 
Bulletin.  Both  are  issued  semimonthly  and  provide  descriptive  entries  and  abstracts  for 
reports  as  well  as  indexes. 

Technical  Report  Copy  Service  provides  users,  upon  request,  copies  of  technical 
reports  either  in  hard  copy  or  on  microfiche. 

Bibliography  Service  offers  users  abstracts  of  selected  documents.  In  addition  to 
a number  of  standard  bibliographies,  DDC  will  make  a computer  search  to  locate  those 
reports  most  pertinent  to  a user’s  research  project. 

Selective  Dissemination  of  Information  (SDI)  is  an  Automatic  Document  Distribu- 
tion service  which  provides  regular  distribution  of  microfiche  copies  of  newly  acquired 
documents  selected  to  match  a user’s  specific  subject-interest  profile. 

DD  1498  Work  Unit  Information  Service  provides  answers  concerning  the  who, 
what,  when,  where,  how,  costs,  and  other  information  of  DoD-sponsored  research  and 
technology  in  progress.  This  information  includes  the  name  and  phone  number  of  the 
person  performing  the  work.  The  system  is  designed  to  enable  the  individual  user  to  de- 
termine the  status  of  research  and  technology  effort  being  performed  in  house  or  under 
contract  or  grant;  historical  information  also  may  be  obtained. 

The  DPI 634  Data  Bank  contains  R&D  program  planning  information  at  project 
and  task-area  levels. 
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The  IR&D  Data  Bank  contains  proprietary  information  on  defense-related  in- 
house  work  from  companies  in  the  DoD-Industry  Independent  Research  and  Develop- 
ment (IR&D)  program.  Because  the  information  is  proprietary,  its  use  is  limited  to 
those  within  DoD.  The  type  of  information  provided  is  similar  to  that  of  the  DD  1498 
Work  Unit  Information  System.  The  IR&D  data  bank  can  be  used  through  channels  other 
than  the  on-line  terminal,  as  explained  in  DSAM  4185.1 1,  the  IR&D  User’s  Manual. 

DDC’s  Referral  Service  provides  information  about  DoD-sponsored  specialized 
sources  of  scientific  and  technical  knowledge.  When  information  exceeding  that  con- 
tained in  DDC  is  required,  this  service  is  used  to  direct  requesters  to  the  National  Refer- 
ral Center  for  Science  and  Technology  and/or  other  potential  sources  of  information. 


VISUAL  SEARCH  MICROFILM  FILE  (VSMF) 

A room  is  set  aside  in  NOSC’s  Research  Library  for  use  of  VSMF  equipment  - 
racks  holding  the  16-mm  roll-film  cartridges,  a microfilm  reader-printer,  and  hard-copy 
indexes  and  user’s  manuals.  Two  main  categories  of  information  are  contained  on  film 
for  retrieval  on  the  reader-printer:  “Product  Information”  and  “Specifications  and  Stand- 
ards.” The  system  is  set  up  for  self-use  after  consultation  with  the  Public  and  Informa- 
tion Services  Librarian. 

Product  Information  Services  consists  of:  Design  Engineering  Service;  Marine  En- 
gineering Service;  Integrated  Circuit  Parameter  Retrieval  Service;  and  Semiconductor 
Parameter  Retrieval  Service. 

The  Design  Engineering  Service  includes  manufacturers’  data  sheets  organized/ 
filmed  so  that  like  items  appear  side  by  side  for  rapid  comparison.  Data  are  indexed  by 
product  description  (more  than  35000  descriptions)  and  manufacturer’s  name  within  the 
following  sections: 

Electrical 

Electronic 

Fluid  Systems 

Instruments 

Materials  and  Fasteners 

Power  Transmission  and  Hardware 

Production  Equipment  and  Services 

The  Marine  Engineering  Service  consists  of  manufacturers’  data  sheets  on  marine- 
unique  products  and  components,  and  is  organized  and  indexed  in  the  same  manner  as 
the  Design  Engineering  Service.  (NOTE:  Although  the  Library  does  not  at  present  pro- 
vide this  service,  it  is  mentioned  here  because  it  is  available  from  the  VSMF  system  in 
the  Design  Engineering  Division.  NOSC.) 

The  Integrated  Circuit  Parameter  Retrieval  Service  allows  IC  device  selection  by 
defined  function,  original  circuit  number,  and  manufacturer’s  device  number,  and  the 
Semiconductor  Parameter  Retrieval  Service  allows  discrete  device  selection  by  electrical 
and  physical  parameters. 
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military  specifications,  MS  drawings,  military  handbooks,  and  all  sections  of  the  American 
Society  for  Testing  and  Materials  (ASTM)  standards. 


The  MIL  Specification  Service  contains  current  military  and  federal  specifications 
and  standards,  JANs,  and  QPLs  indexed  by  document  number,  title,  and  product  descrip- 
tion within  the  following  sections: 

Assemblies 

Electrical 

Instrument 

Mechanical 

Procedures  

General  — 

Two  supplements  to  these  data  also  are  provided:  “Hot  Specs,”  which  are  new,  late- 
release  documents;  and  “Historical,”  those  documents  which  have  been  superseded  by 
documents  in  the  current  file. 

The  MS  Drawing  Service  contains  MS,  AN,  AND,  USAF,  NASA,  and  NAS  draw- 
ings; MIL-D-1000  and  associated  documents;  and  Cataloging  Handbooks  H4-I  and  H4-2; 
with  drawings  of  like  items  filmed  side  by  side  for  easy  comparison.  Indexing  is  by 
document  number,  title,  and  product  description. 

The  Military  Handbooks  Service  contains  all  such  documents,  indexed  by  docu- 
ment number  and  title. 

The  American  Society  for  Testing  and  Materials  Service  contains  all  ASTM  stand- 
ards, indexed  by  document  number  and  subject,  in  the  following  sections: 

Metals 

Construction 

Paint 

Petroleum 

Plastics 

Textiles 

Rubber  and  Electric  Insulating  Materials 

General  Test  Methods 

Miscellaneous 

All  VSMF  Services  are  updated  frequently  and  regularly  to  provide  current  in- 
formation. In  addition,  VSMF  provides  “Extension  99”  - telephone  data  assistance. 
Subscribers  may  use  a direct  telephone  link  to  a staff  of  data  specialists  at  VSMF  head- 
quarters near  Denver  for  assistance  in  making  a search,  for  information  beyond  that  a- 
vailable,  etc. 


LOCKHEED  INFORMATION  RETRIEVAL  SERVICE  DIALOG 


Five  Lockheed  DIALOG  files  are  available  at  the  Research  Library  which  are  of 
particular  interest  to  Center  personnel: 

NTIS  (National  Technical  Information  Service).  Contains  about  400  000  Govern- 
ment-funded unclassified  technical  reports. 

COMPENDEX  (Engineering  Index).  Has  approximately  275  000  citations  in  all 
fields  of  engineering. 

INSPEC  (Institution  of  Electrical  Engineers).  Has  450  000  abstracts  in  areas  of 
electrical  and  electronic,  computer  and  control,  and  physics. 

CHEMCON  (Chemical  Condensates).  Has  over  a million  entries  in  all  fields  of 
chemistry. 

ERIC  (Educational  Resources  Information  Center).  All  fields  of  educational  re- 
search are  covered  by  185  000  citations. 


SYSTEMS  DEVELOPMENT  CORPORATION  ORBIT  II 

Two  ORBIT  II  files  may  be  of  value.  GEO-REF  (American  Geological  Institute) 
contains  about  4 000  000  items  concerning  work  in  geology,  mineralogy,  oceanography, 
geophysics,  geochemistry,  and  geomorphology.  And  SSIE  (Smithsonian  Science  Infor- 
mation Exchange)  has  about  170  000  entries  concerning  scientific  research  projects  that 
are  currently  funded  by  federal  agencies  and  state  and  local  governments,  universities, 
colleges,  and  private  foundations. 


WESTERN  RESEARCH  APPLICATIONS  CENTER  DATACON 

DATACON  accesses  all  NASA  and  NASA-sponsored  research  reports. 

(The  current  capabilities  of  the  three  preceding  data  files  - DIALOG,  ORBIT  II, 
and  DATACON  - include  state-of-the-art  reviews  on  a given  subject;  single  author,  sub- 
ject, or  contract  inquiries;  retrospective  bibliographies  on  a subject  or  concept;  and  cur- 
rently funded  project  searches.) 


ARMY  ECOM  JOINT  ELECTRONIC  TYPE  DESIGNATIONS 

The  Research  Library  has  16-mm  film  cartridges,  prepared  and  maintained  by  the 
Army  Electronics  Command  (ECOM),  containing  data  on  the  parameters  of  all  nomen- 
clatured  equipments  in  the  DoD  inventory.  The  cartridges  are  usable  on  the  VSMF 
reader-printer. 


COMMERCE  BUSINESS  DAILY 

Issues  of  the  Commerce  Business  Daily  are  located  on  the  periodical  shelves  of 
the  Research  Library.  Clues  and  leads  to  ongoing  technology  of  interest,  and  government 
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and  industry  participators,  may  be  found  in  the  sections  on  procurement  invitations  for 
services  and  supplies,  contract  awards,  and  the  solicitation  of  research  and  development 
sources. 

MISCELLANEOUS  PUBLICATIONS 

The  annual  US  Government  Manual  and  the  Navy  Technical  Facility  Register 
(NAVMAT  P3999-1)  contain  information  on  Government  agencies  and  personnel,  the 
latter  being  particularly  useful  for  looking  up  Navy  R&D  centers  to  determine  facilities, 
personnel  responsibilities,  whom  to  contact,  etc.  The  Public  and  Information  Services 
Librarian  also  can  direct  the  investigator  to  other  documents  which  may  be  helpful, 
such  as  specialty  periodicals,  handbooks,  and  annual  buyers’  guides;  society  periodicals 
and  handbooks;  and  vendors’  catalogs. 

GOVERNMENT-INDUSTRY  DATA  EXCHANGE  PROGRAM  (GIDEP) 

The  GIDEP  is  jointly  sponsored  by  the  Army,  Navy,  Air  Force,  Canadian  Mili- 
tary Electronics  Standards  Agency  (CAMESA),  NASA,  FAA,  DSA,  SBA,  and  other  fed- 
eral departments  and  agencies.  GIDEP  provides  automatic  interchange  of  technical  data 
related  to  parts,  components,  and  materials  utilized  in  military  systems.  The  data  are 
primarily  results  from  environmental  testing  conducted  by  participants  who  are  engaged 
in  design,  development,  and  production  of  military  and  aerospace  equipment.  Service 
is  available  without  charge  to  users.  Information  on  GIDEP  may  be  obtained  from: 

Officer  in  Charge  (Code  862) 

Fleet  Missile  Systems  Analysis  & Evaluation  Group 

Corona,  California  91720 

Phone  (714)  736-4677  (Autovon  933^677) 

. 

DoD  INFORMATION  ANALYSIS  CENTERS 

The  Defense  Department  supports  1 8 centers  for  analysis  of  scientific  and  tech- 
nical information.  Each  center  gathers  information  in  its  special  area  of  interest;  re- 
views, analyzes,  evaluates,  synthesizes,  condenses,  and  summarizes  the  information;  and 
provides  it  to  individual  users.  The  centers  produce  critical  reviews,  state-of-the-art 
monographs,  data  compilations,  and  substantive  responses  to  queries.  A charge  is  made 
for  services  to  both  government  and  contractor  users.  Information  on  the  particular 
center  most  likely  to  be  able  to  provide  the  desired  information  may  be  obtained  from 
NOSC’s  Public  and  Information  Services  Librarian. 
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DESIGN  ENGINEERING  DIVISION  SERVICES 


The  Design  Engineering  Division  at  NOSC  and  similar  organizations  at  other  Com- 
mands provide  a number  of  services  to  assist  the  project  manager;  these  include: 

Searches  of  the  Navy’s  Maintenance  Data  Collection  System  (MDCS).  MDCS  is 
useful  in  analyzing  current  equipment  performance  and  reliability  and  can  help 
establish  design-to-cost  parameters. 

Assistance  in  project  planning  and  especially  in  developing  tailored  specifications, 
tailored  environmental  testing  criteria,  tailored  support  methodologies,  selection 
and  screening  criteria  for  existing  commercial  equipments,  and  life-cycle  cost 
estimates. 

Installation  planning. 

Assistance  in  parts  selection  especially  using  military  specifications  and  the  Na- 
tional Stock  System. 

The  Design  Engineering  Division  also  performs  packaging  design,  provides  drafting  services, 
and  coordinates  printed  circuit  layout  and  fabrication. 

At  NOSC,  the  Design  Engineering  Division  also  provides  the  VSMF  service. 


MILITARY  PARTS  CONTROL  ADVISORY  GROUP  (MPCAG) 

The  MPCAG  provides  free  parts  selection  advice  supported  by  an  extensive  staff 
and  computer  facilities.  Telephone  numbers  and  addresses  to  obtain  assistance  are  listed 
in  MIL-STD-1631  A. 


PERFORMANCE  PARAMETERS 


There  are  two  aspects  to  the  specification  of  performance  - the  degree  of  func- 
tional proficiency  to  be  achieved  and  the  severity  of  the  environment.  Traditionally, 
the  specification  approach  has  been  to  obtain  as  much  performance  as  possible  and  to 
assume  the  environments  called  out  in  military  standards  and  specifications;  this  approach 
drives  program  costs  upward  and  creates  unnecessary  technical  risks.  The  approach  rec- 
ommended herein  is: 

1.  Specify  only  those  functions,  modes,  and  characteristics  completely  which 
are  necessary  to  meet  the  operational  requirements. 

2.  Specify  environments  which  realistically  reflect  the  actual  usage  environ- 
ments. 

Military  specifications  normally  reflect  a “worst  case”  environment.  This  is  com- 
posed of  the  most  extreme  cases  of  a multitude  of  environments  which  are  usually  grossly 
in  excess  of  the  extremes  normally  encountered  in  any  one  usage  environment.  Table  IV-2 
summarizes  the  environmental  parameters  which  should  be  considered;  appendix  A pro- 
vides more  detailed  environmental  criteria.  Test  and  evaluation  planning  must  be  struc- 
tured with  the  specified  environment  as  a baseline  (see  chapter  XIX). 

In  order  to  establish  those  characteristics  which  can  be  considered  “necessary,” 
the  following  definitions  are  useful: 

Essential  Those  characteristics  dictated  by  the  system 

mission 

Less  than  essential  Characteristics  which  contribute  usefully 

toward  the  system  mission 

Nonessential  Everything  else 

Necessary  characteristics  can  now  be  defined  as  all  essential  characteristics  plus 
only  those  less-than-essential  characteristics  the  provision  of  which  will  not  complicate 
operation  or  maintenance  and  the  utility  of  which  outweighs  the  cost  of  providing  them. 
When  in  doubt,  leave  it  out!  Some  characteristics  may  be  essential  to  one  technical  ap- 
proach and  nonessential  to  another;  they  are  specified  only  when  the  technical  approach 
is  specified.  System  specifications  generally  do  not  specify  a technical  approach  unless 
only  one  approach  is  clearly  acceptable;  equipment  specifications  may  direct  a particular 
technical  approach,  especially  when  multiple  approaches  are  pursued  in  parallel. 

In  determining  the  specification  of  a characteristic,  quantity  must  be  considered 
as  well  as  quality.  “The  receiver  must  have  high  sensitivity”  - how  much  is  high? 

Most  specified  parameters  fall  into  a gray  region  where  a value  is  not  known.  Several 
techniques  are  available  which  can  be  applied  to  resolve  parametric  unknowns. 

Figure  of  merit 

System  effectiveness  model 

Operations  analysis 

Value  enginecring/cost-benefit  analysis 

Laboratory  testing 
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Exploratory  development 

Analysis  of  similar  equipment 

A system  effectiveness  model  is  useful,  since  it  incorporates  performance  parame- 
ters, thus  allowing  tradeoffs  between  performance  and  support.  System  effectiveness 
is  defined  as  a product  of  availability,  dependability,  capability,  and  utilization.  Availa- 
bility and  dependability  are  support  parameters  and  are  discussed  further  in  the  next 
section.  Utilization  is  an  adjustment  or  degradation  factor  employed  in  the  event  that 
the  system  occasionally  is  used  under  conditions  of  stress  greater  than  the  design  environ- 
ment; otherwise,  it  is  set  to  1.  Capability  is  the  probability  of  a successful  mission,  given 
that  the  system  is  operated  within  specifications  and  that  no  failures  occur.  Capability 
is  a routinely  applied  concept  in  ordnance  systems  where  it  is  the  product  of  the  proba- 
bility of  hit  and  the  probability  of  kill  given  hit. 

Figure  of  merit  (FOM)  is  a nonprobabilistic  measure  of  capability  which  can  be 
readily  developed  for  most  types  of  electronics  and  converted  to  Capability  (C)  through 
the  following  formula;  FOM  (measured)  - L _ ^ 

FOM  (specified)  - L 

FOM  can  be  established  by  determining  which  parameters  directly  interplay  in  the  sys- 
tem performance  and  determining  a common  unit  of  measure.  The  unit  of  measure  may 
be  simple  (eg,  dB)  or  complex  (bits/ms-dB).  Environmental  factors  (such  as  atmospheric 
noise)  and  human  factors  (such  as  operator  proficiency)  may  be  assigned  weights  and 
included  in  the  commutation.  Analysis  supported  by  laboratory  tests  is  usually  sufficient 
to  establish  a predicted  FOM  (FOM  (specified))  which  ensures  satisfactory  system  opera- 
tion and  a limiting  figure  (L)  below  which  the  system  will  not  operate.  The  capability 
figure  established  by  utilizing  a figure  of  merit  derivation  is  not  necessarily  accurate,  but 
it  is  a sufficiently  useful  concept  to  warrant  consideration.  System  effectiveness  can  be 
converted  to  cost-effectiveness  by  dividing  by  the  total  life  cost. 
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TABLE  IV-2.  EQUIPMENT  ENVIRONMENTAL  TESTS  AND  REQUIREMENTS. 

GENERAL  REQUIREMENTS 
tsignator  Environment 

1 . Temperature  (operating  and  nonoperating) 

2.  Temperature-altitude 

3.  Humidity 

4.  Thermal  shock 

5.  Vibration 

6.  High  impact  shock 

7.  Transport  shock 

8.  Repetitive  shock 

9.  EMC  (interference  and  susceptibility) 

10.  EMP 

1 1.  Electrical  transients  (voltage  and  frequency /long  term  and  short  term) 

12.  Lightning 

13.  Magnetic  field 

14.  Acoustic  noise  (airborne  and  structureborne) 

15.  Inclination 

16.  Radiation 

17.  Nuclear  air  blast 

18.  Gun  blast 


Icing  with  wind 

Rain  and  snow,  snow  loading 

Sunshine 

Degree  of  enclosure 
Dust 

Salt  fog,  spray,  solution 
Damaging  (corrosive)  atmospheres 
Explosive  atmosphere 
Fungus 

Maintainability /bench  handling 

Reliability  (burn-in,  confidence,  indexing,  accelerated  life,  failure-mode  analysis) 
Combined-environment  testing  (temperature-humidity-vibration-electrical  transients 
on/off  cycling) 

On/off  cycling 

Acoustic  susceptibility  (in  high-noise  environments) 

Water  impact/hydrostatic  pressure 

Underwater  explosion  (for  hull-mounted  equipments  only) 

Drop  test 

Equipment  special  environments 
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TABLE  1V-2.  (Continued). 


SPECIAL  REQUIREMENT  CATEGORIES 
Category  Environments  Notes 

Vital  equipments  10,16,17  16  and  17  for  exposed  equipments 

6 operating 

Semivital  equipments  6 nonoperating  (normal  operating  before 

and  after  shock) 

Nonvital  equipments  6 safety  criteria 

Exposed  equipments  12,18,19,20,21,22,23,24 

Sheltered  equipments  23 

Standard  requirements  1,3,5,6,9,11,14,29,30,31,3237 


APPLICATION  REQUIREMENTS1 


plication  Environments  General  Equipment  Spec 


Test  Spec 


Notes 


Shipboard  8,13,15,25,26,  MIL-E-16400  MIL-E-16400  for  (5)  MIL-STD-167 

27,33,34,35  for  (6)  MIL-S-901 

Environmental  Interfaces 
MIL-STD-1399 

for  (1 5)  per  MIL-E-16400  except  30° 
(operating)  and  60°  (without  damage  or 
spillage)  for  submarines 


Shore  (fixed) 

33 

MIL-E-16400 

MIL-E-16400 

for  (5)  .5g  to  50  Hz 

for  (6)  MIL-S-901  may  be  waived 

Shore  (mobile, 

transportable, 

vehicular) 

2,4,7,15 

MIL-E-4158 

l 

MIL-STD-8 10 
MIL-STD-1 69 
MIL-STD-1 70 
MIL-STD-2 10 
MIL-STD-1 474 
MIL-D-13570 

Airborne 

2,4,7,26,27,28 

MtL-E-5400  Prop,  Jet,  and 
Helo  Aircraft 

MIL-E-8189  Missiles,  Boost- 
ers, and  Allied  Vehicles 
MIL-E-1 1991  Guided  Missiles 

MIL-STD-8 10 
MH.-T-5422  (Navy) 
Gen  Equip  Spec 

Portable 

36  plus  applica- 
ble general  ap- 
plication 

Same  as  general  application 

Same  as  general  ap- 
plication plus  detail 
spec. 

- 

Space  f 

2,4,7,8,16,33 

MIL-STD-1 540 

MIL-STD-1 540 

for  (9)  MIL-STD-1 541  equipments  are  all 
considered  vital 

Test  equipment  7,24,25,27,28,33  MIL-T-28800 
Amphibious 

MIL-STD-8 10 

Shipboard  and  Shore  (mobile)  combined 

Torpedoes 

22,24,25,28,34 

Ml  L-T- 18404  , 

MI  L-T- 18404 

(37)  includes  acceleration 

Shipboard 
fire  control 

2,8,13,15,25, 

26,2733 

MIL-F- 18870 

MIL-T-18870 

J 9 

Same  as  Shipboard  except  as  modified 
by  MIL-T-18870.  (2)  is  nonoperating 

transportation  test. 


' Environmental  requirements  should  consider  standard  sheltered  or  exposed,  and  vital-semivital-nonvital  requirements  in 
addition  to  those  listed. 
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SUPPORT  PARAMETERS 

The  previous  section  stressed  performance  — the  capability  of  the  end  product 
to  do  the  desired  job.  Equally  important  to  the  user  is  the  supportability  of  the  end 
product.  Equipments  which  are  not  up  and  operating  are  worse  than  no  equipment  at 
all;  the  down  equipment  is  not  doing  its  intended  job,  no  matter  how  much  capability 
was  designed  into  it.  Furthermore,  nonsupportable  equipments  consume  valuable  re- 
sources in  money  and  manpower  to  effect  their  repair.  Note  that  of  the  four  factors 
which  make  up  system  effectiveness,  two,  capability  and  utilization,  are  performance 
related,  and  two,  availability  and  dependability,  are  support  related.  Besides  playing  a 
major  role  in  system  effectiveness,  support  parameters  describe  the  support  phase  costs. 
Since  support  phase  costs  are  a major  portion  of  the  total  cost  of  ownership,  supporta- 
bility and  affordability  are  incontrovertibly  intertwined. 

Underlying  availability  and  dependability  are  the  more  familiar  concepts  of  re- 
liability, maintainability,  and  downtime  and  the  measures  associated  with  each.  Since 
some  of  the  terms  involve  technical  shades  of  meaning,  figure  IV-3  has  been  provided. 

Traditionally,  reliability  and  maintainability  have  been  the  primary  factors  con- 
sidered in  supportability  design.  Both  can  be  stated  as  requirements  for  which  tests 
can  be  constructed.  The  usual  measures  of  reliability  and  maintainability  are  MTBF 
and  MTTR,  respectively,  which  are  combined  in  inherent  availability  (see  figure  IV-3, 
formula  1).  Inherent  availability  is  a specification  mechanism  which  constrains  the 
tradeoffs  between  MTBF  and  MTTR  where  it  is  desired  to  achieve  values  in  excess  of 
minimum  specified  values.  Inherent  availability  ^nnot  be  used  to  predict  the  equip- 
ment availability  in  use;  it  is  a < pecification  tool  only  and  must  be  treated  as  such. 

MTBF  and  MTTR  are  often  considered  independent  of  the  operating  environment; 
this  is  a gross  fallacy.  Reliability  depends  on  two  major  factors  ( 1 t the  selection  of 
reliable  parts  (those  which  are  produced  by  mature  processes  under  established  quality 
control)  and  (2)  the  use  of  reliable  design.  Performance  must  influence  design  com- 
plexity, parts  counts,  and  other  factors  affecting  reliability;  however,  the  reliable  design 
will  always  utilize  parts  within  their  ratings  even  under  specified  stresses  (temperature, 
humidity,  electrical  transients,  vibration,  etc)  and  will  allow  for  variations  in  part  value 
arising  from  the  production  process  and  from  aging  effects.  Maintainability  depends  on 
the  fault  isolation  characteristics,  accessibility  of  parts,  testability  of  functions,  and 
ease  of  control  adjustment  inherent  to  the  design. 

Since  the  operating  environment  can  usually  be  readily  specified  (or  readily  over- 
specified  through  blanket  reference  of  militaiy  specifications),  the  conscientious  engineer 
can  formulate  reliable  designs  (ie,  within  the  state  of  the  art).  The  maintenance  environ- 
ment is  usually  not  widely  recognized  by  either  the  specification  writer  or  the  designer 
responding  to  a specification.  Nevertheless,  it  is  at  least  as  important  to  specify  and  de- 
sign to  the  maintenance  environment  as  to  the  operating  environment;  otherwise,  the 
equipment  will  experience  limited  availability,  excessive  support  costs,  and  reduced  ser- 
vice life. 

Most  of  the  maintenance  environment  can  be  described  as  a level  of  maintenance 
capability  at  the  user  organization.  The  remaining  portion  considers  the  capabilities  of 
intermediate  and  depot  facilities.  Table  IV-3  describes  levels  of  organization  mainten- 
ance capability;  table  IV-4  suggests  some  parameters  which  are  within  these  capabilities. 
Equipments  should  never  be  designed  in  excess  <-»f  the  capabilities  of  the  least-capable 
platform;  table  IV-5  is  provided  for  guidance.  Wherever  possible,  shipboard  equipments 
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(Source:  ILS  Implementation  Guide  - NAVMAT  P-4000) 


Definitions 

1.  Availability  (A):  The  probability  that  an  item  will  be  ready  to  perform  within  specification  at  the  start  of  a mission. 

2.  Dependability  (D):  The  probability  that  an  item  will  be  able  to  perform  a complete  mission  within  specification  and,  should  a 
failure  occur  during  that  time,  can  be  restored  to  operation  within  specification  within  the  allowed  downtime. 

3.  Reliability  (R ) : The  probability  that  an  item  will  be  able  to  perform  a complete  mission  within  spec  i.zation,  given  that  it  is 
operational  at  the  start  of  the  mission.  Alternately,  the  probability  that  an  item  will  operate  within  specification  for  a given 
period  of  time. 

4.  Maintainability  (M):  The  probability  that  an  item  can  be  restored  to  operation  within  specification  within  a given  downtime. 

5.  Supportability  (S):  The  probability  that  all  the  elements  necessary  to  effect  the  repair  of  an  item  will  be  available  within  a 
specified  time. 

6.  MTBF:  Mean  time  between  failures  (or  before  failure). 

7.  MTBUR:  Mean  time  between  unscheduled  removals.  (Used  for  placeable  assemblies.) 

8.  MTBCM  or  MTBCMA:  Mean  time  between  corrective  maintenance  actions. 

9.  MTBPM.  Mean  time  between  preventive  maintenance  actions. 

10.  MTBM:  Mean  time  between  maintenance  actions. 

11.  MTTR:  Mean  time  to  repair. 

12.  MCT:  Mean  corrective  maintenance  time. 

13.  MDT:  Mean  downtime. 

14.  Failure:  Any  item  condition  in  which  the  item  is  unable  to  operate  within  specification. 

a.  Relevant  Failure:  A failure  with  which  'the  item  cannot  perform  its  specified  mission. 

b.  IMonrelevant  Failure:  A failure  which  causes  degraded  item  performance  but  does  not  prevent  the  item  from  performing 

its  specified  mission. 

c.  Partial  Failure:  A failure  of  one  or  more  modes  of  operation  which  are  not  critical  to  the  item's  specified  mission. 

15.  Ready  time:  The  time  during  which  the  item  is  capable  of  operation  but  not  in  use. 

16.  MCMH:  Mean  corrective  maintenance  man-hours. 

Figure  IV-3.  Support  parameter  definitions  and  formulae. 
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1.  A:  (inherent  availability)  = 

' MTBF  + MTTR 


2.  AQ  (operational  availability)  - 


MTBM  + ready  time 
MTBM  + MDT  + ready  time 


a.  (discounting  preventive  maintenance  by  assuming  scheduling  of  PM  to  not  interfere  with  possible  use) 

. _ MTBCM  + ready  time 

0 MTBCM  + ready  time  + MDT 

b.  (assuming  100%  utilization)  A_  * —■ 

° MTBCM  + MDT 

3.  a.  D = Rm  + M(1-Rm)  (Rm  = mission  reliability ) 


b.  D = R„ 


(no  allowable  downtime) 


4.  a.  Rm  (mission  reliability)  = e ^y---  (T  = length  of  mission) 
b.  R0  (operating  reliability)  = e ~MTgC^ 

5.  MDT  = MTTR  + MDT  (admin)  + MDT  (parts)  + MDT  (assistance) 

6.  MDT  (admin)  = FRD  (fault  recognition  delay)  + MDT  (test  equipment) 

+ MDT  (documentation)  + SRT  (supply  requisition  time) 

+ CB  (coffee  breaks  and  other  periods  of  technician  nonavailability) 

7.  MTTR  + Time  required  to  procedurally  fault  isolate  to  a replaceable  assembly 

+ time  necessary  to  access  and  replace  the  faulty  assembly 

+ time  to  reassemble  and  check  out  the  equipment  including  realignments  as  necessary. 


Specification  Terms: 


1.  MTTRQ:  MTTR  at  the  organization  level 

2.  MTT R | MTTR  at  the  intermediate  level; 

no  more  than  2X  MTTRQ 

3.  MTTRq:  MTTR  at  the  depot  level;  no  more 

than  2X  MTTRj  or  as  determined 
by  * life-cycle  cost  model/level  of 
repair  model 

4.  Annual  preventive  maintenance  time 

5.  Cost  per  repair  action 


Table  IV-4 

Table  IV-4  t expected  # actions 

Operational  requirement 

Determined  by  a life-cycle  cost  model 

Vital  equipments  0.99 

Semivital  equipments  0.90 

Nonvital  equipments  0.75 


Figure  1V-3.  (rontinued) 
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s recognized  in  NAVSUPINST  4423. 14A,  Enclosure 


should  be  designed  for  level  2 capabilities  with  the  primary  maintenance  load  within  the 
abilities  of  an  E-3  technician  and  with  the  most  difficult  tasks  requiring  only  an  E-4.  The 
designer  must  be  provided  with  constraints  through  the  equipment  specification.  Con- 
straining requirements  should  include  MTTR  at  the  organizational,  intermediate,  and  depot 
repair  levels  and  a total  annual  preventive  maintenance  time  ceiling.  A target  mean  cost 
per  repair  is  also  desirable. 

Notice  that  there  are  two  kinds  of  reliability  (see  fig  IV-3,  formula  4).  Mission 
reliability  is  based  in  the  definition  of  reliability  (fig  IV-3,  definition  3).  However,  be- 
cause of  redundancy,  multiple  modes  of  operation,  gracefully  degraded  operation,  voting 
techniques,  and  multiple-mission  requirements,  many  equipment  conditions  can  occur 
which  require  maintenance  without  constituting  an  equipment  failure  (in  the  MIL-STD- 
781  sense).  Therefore,  the  specification  should  state  requirements  for  MTBF  and  MTBM. 
The  MTBF  requirement  is  generally  only  important  when  a specific  mission  time  is  of 
interest.  MTBM  is  the  requirement  that  affects  the  support  costs  directly. 

Returning  to  availability  and  dependability,  it  can  be  seen  that  reliability  and  main- 
tainability do  play  an  important  role  in  these  factors.  However,  a missing  term  is  down- 
time. Downtime  consists  of  active  repair  time,  administrative  time,  and  time  awaiting 
parts  and  outside  assistance.  Mean  downtime  is  the  normal  measure  of  downtime  per 
maintenance  action.  The  tightening  DoD  budgets  of  recent  years  have  increased  the  im- 
pact of  downtime,  especially  on  operational  availability  (see  fig  IV-3,  formula  2).  In 
recent  equipment  improvement  programs,  MTBCM  has  been  increased  fivefold  and  MTTR 
halved,  but  operational  availability  has  dropped  from  0.75  to  0.30  because  of  MDT  deter- 
ioration. Referring  to  figure  IV-3,  formulas  5 and  6,  it  can  be  seen  that  many  of  the 
factors  affecting  MDT  are  not  directly  related  to  design;  nevertheless,  design  features  can 
have  a tremendous  impact  on  the  various  forms  of  downtime.  In  order  to  provide  a 
meaningful  specification  of  operational  availability,  MDT  must  be  realistically  specified  on 
the  basis  of  design  characteristics  and  the  “facts  of  life”  in  shipboard  routine.  Figure  IV-4 
provides  suggested  specification  values  for  downtime.  It  is  useful  to  specify  operational 
availability  while  specifying  only  minimum  acceptable  values  for  MTBF,  MTTR,  and 
MTBM;  this  allows  the  designer  more  latitude  while  constraining  the  design  more  closely 
to  the  desired  end  product.  Tests  can  be  provided  for  the  reliability  and  maintainability 
parameters.  The  remaining  values  are  assigned  in  accordance  with  the  provision  of  certain 
features  (as  in  fig  IV-4),  and  the  expected  availability  is  computed  to  check  conformance 
with  the  specification  requirements.  The  formulae  to  be  used  in  this  computation  must 
be  stated  in  the  specification.  Dependability  is  never  specified;  the  dependability  figure 
is  derived  for/from  a system  effectiveness  model  by  utilizing  figure  IV-3,  formulas  3b  and 
4a. 


TABLE  IV-5.  CURRENT  MAINTENANCE  CAPABILITIES 
OF  MAJOR  SHIP  CLASSES. 


Capability  Class 

6.  CV.  CVA,  CVN,  CG.  CGN,  LPH.  LHA.  SSBN.  LCC.  AD.  AS.  AR 

5.  DD.  DDG,  LPD,  SSN.  AGF 

4.  FF,  FFG-I,  LSD.  LST,  AOE,  AOR.  AFS.  AE 

3.  SS.  LKA,  LPA,  AO,  AF,  FFG-7,  ASR 

2.  PG,  PHM.  MSO,  ATF,  ARS.  ATS 


MOT  « MTTR  + MDT  (Parts)  ♦ MOT  (Assist)  + MDT  (Admin) 
MOT  (Parts) 


MDT  (Parts)  * (part  supply  time  in  hours  for  the  i,h  part)  X \i 


Part  Supply  Time 

0.02  h 
0.07  h 
0.5  h 
6.0  h 
720.0  h 
4320.0  h 
12  960.0  h 


(failure  rate  for  the  i,h  part)  X MTBF  (of  the  equipment) 

Part  Description  of  Replacement  Item 
built-in  spares 

battle  spares  (stocked  in  separate  spares  kit) 

preferred  parts  (high-usage  standard  parts  per  MIL-STD-242) 

standard  parts  (per  MIL-SPEC) 

life-cycle  stocked  parts  and  nonstandard  controlled  parts 
uncontrolled  parts  (not  system  peculiar) 
uncontrolled  parts  (system  peculiar) 


(based  on  1/2  hour  for  parts  on-board,  24  hours  for  parts  not-on-board  but  expected  to  be  on-board  ships-in- 
company,  30  days  for  parts  not-on-board  but  in-stock.  1 80  days  for  parts  not-on-board.  not-in-stock  (commonly 
available).  540  days  for  parts  not-on-board,  not-in-stock  (not  commonly  available)) 

NOTE:  A controlled  part  is  a nonstandard  part  which  is  fully  provisioned:  a life-cycle  stocked  part  is  a system-peculiar 

part  which  is  stocked  in  sufficient  quantity  to  support  the  expected  life  of  the  system  (long-lead-time  items  may 
be  included  if  they  are  fully  provisioned).  Uncontrolled  parts  include  all  items  which  are  not  standard  or 
controlled. 

MDT  (Assistance) 

Task 


Organizational  IM  Repair 
Repair  (on  board) 

0 15  min 

0 24  h 

0 120  h 


IM  Repair 
(mobile  assist) 

IM  Repair 
(off  ship) 

Depot 

Repair 

Equipment 
Turnaround 
to  on-board  pool 

Equipment 
Turnaround 
to  IM  pool 

36  h 

48  h 

14  days 

30  min 

36  h 

96  h 

120  h 

21  days 

30  min 

48  h 

240  h 

240  h 

30  days 

30  min 

96  h 

12) 

(3)  (4) 

(3)  14) 

(5)  (3) 

(3) 

Equipment 


14  days 
14  days 
14  days 


Equipment 

Turnaround 


14-30  days 
14-30  days 
14-30  days 

(3)  (6) 


Notes.  (1)  by  definition 

(2)  plus  MDT  (parts)  as  computed 

(3)  MDT  (parts)  equals  zero 

(4)  highly  variable;  depends  on  ship's  schedule 

(5)  highly  variable;  depends  on  organizational  work  loads 

(6)  Depends  on  warranty  provisions 

(7)  Applies  to  aircraft  carriers  and  tenders  only 

Equipment  MOT  (Assistance)  will  be  required  as  follows: 

(1)  Depot  and  equipment  turnaround  as  determined  by  the  equipment  support  concept 
do  not  add  in  for  those  items  for  which  redundancy  is  provided  within  the  system. 

(2)  IM  repair  will  be  required  in  direct  proportion  to  the  percentage  of  maintenance  tasks  which  are  beyond  the  organization  s level 
of  maintenance  capability  (tables  I V-3  and  IV-51  assuming  no  special  training,  test  equipment,  or  tools  are  required.  If  specie 
training  is  required,  add  (2  X number  of  weeks  training  required  v C)  %.  where  C = 1 for  nonvital,  2 for  sent, vital,  and  5 for  vital 
equipments.  If  special  tools  or  test  equipment  are  required,  add  0.15  X percent  of  tasks  requiring  these  items. 

Figure  IV4.  Mean  downtime  (MDT)  specification  values. 


< 


MDT  (Admin) 


MDT  (Admin)  - FRD  + MDT  (TE)  + SRT  + MDT  (Doc)  + CB 

Delay 

FRD  (Fault  Recognition  Delay)  0.02  h 


Category 

Faults  automatically  detected  and 
alarmed 


= E (category  delay)  X (%  predicted 


faults  in  category) 

0.03  h 

Operator  apparent  faults 

0.25  h 

Primary  mode  faults,  not  operator 
apparent 

4h 

Secondary  mode  faults 

(Faults  detectable  only  by  preventive  maintenance  — 
interval  between  applicable  PMS  actions) 

use  0.1  the 

MDT  (TE)  (downtime  for  tools  and  test  equipment) 

* X (category  delay)  X (%  predicted 
faults  in  category) 

0 

Faults  requiring  built-in  test 
equipment  and  tools 

0.8  h 

Faults  requiring  standard  tools 
and  special -purpose  test 
equipment 

0.25  h 

Faults  requiring  general-purpose 
test  equipment 

0.33  h 

Faults  requiring  special  tools 

SRT  (Supply  Requisition  Time) 

0 

Faults  requiring  built-in  or  battle 
spares 

* X (category  delay)  X (%  predicted 
faults  in  category) 

0.33  h 

All  other  faults 

MDT  (Doc)  (downtime  for  documentation) 

0 

% Tech  manuals  built  into  the 

equipment 

0.05  per  volume  of  tech,  manual 

CB  (coffee  breaks  and  other  periods  of  technician  nonavailability) 

Vital  equipment  0.03  h + 0.08  h/2  h of  MTTR 

Semivital  equipment  0.03  h + 0.08  h/1  h of  MTTR 

Nonvital  equipment  0.08  h + 0.17  h/1  h of  MTTR 

Figure  IV-4.  (Continued) 


EQUIPMENT  PARTITIONING 


The  partitioning  of  a system/equipment  plays  an  increasingly  important  role  as 
the  equipment  design  matures;  this  action  becomes  a major  issue  in  the  transition  to  pro- 
duction (see  chapter  VI).  Ultimately,  the  equipment  partitioning  largely  determines  the 
maintainability/repairability  and  the  availability  of  logistics  support  of  the  end  item.  The 
cost  of  supporting  the  equipment  can  be  greatly  influenced  by  the  implementation  of 
standardization;  however,  equipments  which  are  procured  in  large  quantities  will  benefit 
less  from  standardization,  since  equipment-peculiar  items  will  be  procured  in  sufficient 
quantities  to  attain  many  of  the  cost  advantages  of  standard  items.  In  general,  the  fol- 
lowing guidelines  should  be  followed  in  partitioning  actions: 

Small-Quantity  Items  Large-Quantity  Items 


1.  Partition  for  maximum  use  of  off-the 
shelf  items  and  minimize  equipment- 
peculiar  items. 

2.  Partition  for  maximum  internal  stand- 
ardization (ie,  minimize  the  number  of 
different  items). 

3.  Utilize  technologies  which  are  suffi- 
ciently mature  for  producibility  and 
sufficiently  state  of  the  art  to  remain 
available  through  the  projected  equip- 
ment life. 


1 . Partition  for  ease  of  maintenance  and 
repair,  providing  functions  which  are 
easily  identified  and  isolated  to  a replace- 
able assembly. 

2.  Partition  to  obtain  the  most  economic 
level  of  replacement;  use  a LOR  analysis 
where  appropriate  (see  MIL-STD-1390). 

3.  Points  (2)  and  (3)  for  small-quantity 
items  also  apply. 


Large-quantity  items  are  those  items  for  which  one-time  manufacturing  costs  amor- 
tized over  the  production  quantity  are  small  (under  5%  of  the  unit  cost).  Figure  IV-5  can 
be  used  as  guidance  in  this  determination. 

The  resulting  equipment  partitioning  should  be  integrated  into  the  project  work 
breakdown  structure  to  serve  as  a basis  of  cost  planning  for  management  control,  and  to 
exercise  tradeoff  models  (as  for  life-cycle  costing).  (See  chapter  III.) 
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QUANTITY 


INTERMEDIATE  = LARGE  WHEN  EACH 
INTERMEDIATE  = SMALL  WHEN  EACH 
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V.  VALIDATION  PHASE 

The  validation  phase  serves  as  a vital  checkpoint  on  decisions  made  during 
the  conceptual  phase  and  provides  for  a smooth  transition  into  the  devel- 
opment/procurement cycle.  This  chapter  cites  the  tasks  necessary  to 
achieve  these  ends.  The  tasks  cannot  be  slighted  without  adversely  affect- 
ing the  equipment  total  life  costs. 


INTRODUCTION 

The  purpose  of  the  conceptual  phase  is  to  spawn  one  or  more  technical  approaches 
within  the  feasible  limits  of  technology.  Each  technical  approach  embodies  technical  re- 
quirements which  form  the  system  specification.  How  does  one  know  that  those  technical 
requirements  respond  to  the  operational  requirements?  The  purpose  of  the  validation  phase 
is  to  provide  an  answer  to  that  question  and  to  provide  tolerances  for  the  technical  para- 
meters. To  these  ends,  a validation  phase  is  mandatory;  however,  its  start  and  finish  may 
be  deeply  imbedded  in  the  conceptual  phase  and  the  transition  to  production  with  no 
clearly  defined  boundary.  — — " 

The  first  task  of  the  validation  phase  relates' the  technical  requirements  to  the 
operational  requirements.  There  are  three  ways  this  may  be  accomplished: 

Analysis 

Simulation 

Advanced  development 


Analysis  is  a suitable  method  only  when  the  technical  approach  has  been  proved  by  past 
experience  to  adequately  address  the  operational  requirements.  Simulation  entails  risks 
of  uncertainty  which  increase  exponentially  with  the  complexity  of  that  being  simulated. 
Advanced  development  provides  the  greatest  degree  of  certainty.  Validation  by  develop- 
ment is,  however,  rather  expensive  and  time  consuming  compared  to  simulation  techniques 
because  an  advanced  development  model  (ADM)  must  be  designed,  built,  and  tested.  When 
the  technologies  being  employed  are  relatively  novel,  development  is  the  only  satisfactory 
method;  thus,  the  atomic  test  program  was  essential  to  the  development  of  atomic  weapons. 
On  the  other  hand,  it  is  foolish  to  build  one  very  complex  item,  say  an  aircraft  carrier,  to 
find  out  whether  that  one  item  is  worth  building,  or  to  start  a war  to  obtain  a realistic 
wartime  environment.  Usually,  a balanced  program  utilizing  both  simulation  and  develop- 
ment is  necessary. 

For  relatively  simple  items,  the  ADM  and  the  conceptual  phase  feasibility  demon- 
stration might  be  combined  Jor  efficiency.  For  complex  items  required  in  small  numbers, 
the  ADM  can  be  combined  with  the  transition  to  production.  In  all  other  cases  where 
development  is  indicated,  the  ADM  should  be  a stand-alone  phase.  Furthermore,  if  the 
tests  of  the  first  ADM  are  not  satisfactory,  successive  ADMs  may  be  required  until  the 
tests  are  OK. 

Besides  the  functional/technical  parameters,  it  is  also  necessary  to  validate  the 
suitability  of  the  design  concept  to  the  usage  environment.  This  is  the  second  task  of  the 
validation  phase.  Since  it  is  seldom  possible  to  evaluate  all  the  factors  which  make  up 
“suitability”  in  a live  environment,  it  is  generally  necessary  to  validate  the  design  through 
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analysis  and  simulation.  The  necessary  tasks  are  incorporated  into  the  Integrated  Logistics 
System  (ILS)  tasks  outlined  in  the  ILS  Implementation  Guide  (NAVMAT  P4000) 
and  chapter  X.  Formal  design  analyses  may  or  may  not  be  required  depending  on  the 
complexity  of  the  design  and  the  familiarity  of  the  designer  with  the  usage  environment; 
where  the  equipment  is  complex  or  the  designer  naive  to  the  real-life  user  environment, 
specialists  should  be  integrated  into  the  design  team.  Some  of  the  more  important  tasks 
are  embodied  in  the  following  formal  analyses: 

Failure  Modes  and  Effects  Analysis  (FMEA) 

Maintainability  Analysis/Demonstration 
Hazard  Modes  and  Effects  Analysis  (HMEA) 

Human  Engineering  Analysis 
Parts  Selection  Review 

Metrology  Analysis  and  Test  Point/Provisions  Review 

These  tasks  are  really  basic  engineering  practices  which  need  not  be  emphasized  through  a 
formal  program  but  which  cannot  be  overlooked.  They  are  often  repeated  and  refined 
many  times  as  the  design  matures.  Their  early  consideration  is  important  because  exten- 
sive design  changes  may  be  required  to  correct  the  deficiencies  they  reveal.  The  earlier 
these  changes  are  incorporated,  the  less  they  impact  on  schedule  and  cost. 

Documentation  is  essential  in  the  validation  phase  in  order  to  provide  a baseline 
for  the  activities  which  follow.  Engineering  drawings  are  particularly  important  to  pro- 
viding design  traceability  through  the  changes  which  inevitably  occur;  change  control 
procedures  should  be  implemented  to  ensure  that  the  documentation  accurately  reflects 
the  design. 


VALIDATION  PHASE 

This  is  the  phase  in  which  the  major  program  characteristics  (technical,  cost,  and 
schedule),  through  extensive  analysis  and  hardware  development,  are  validated  and  is  often 
identified  with  advanced  development.  It  is  preferred  to  rely  on  hardware  development 
and  evaluation  rather  than  paper  studies  alone,  since  this  provides  a better  definition  of 
program  characteristics,  higher  confidence  that  risks  have  been  resolved  or  minimized,  and 
greater  confidence  in  the  ultimate  outcome.  Nevertheless,  effectiveness  analysis  plays  a 
critical  role  since  it  provides  a structured  vehicle  for  interrelating  and  evaluating  the  infor- 
mation developed  as  a result  of  hardware  development,  and  organizes  information  in  a 
form  suitable  for  updating  the  Development  Concept  Paper  (DCP),  and  for  DSARC  re- 
view leading  to  the  decision  to  enter  full-scale  development.  In  an  idealized  case,  this 
phase  ends  when  a brassboard  model  has  been  demonstrated  successfully. 

The  validation  phase  is  a period  of  major  concern  to  the  project  manager,  although 
the  burden  of  proving  performance  and  responsiveness  rests  on  the  contractor  (private 
industry  or  government  laboratory)  who  has  been  selected  as  a qualified  participant  essen- 
tially on  the  basis  of  proposals. 

In  many  respects  the  application  of  the  system  value  methodology  to  validation 
parallels  its  application  in  the  conceptual  phase.  There  are,  however,  some  significant  dif- 
ferences. In  time  sequence,  the  system  value  concept  is  applicable  during  the  conceptual 
phase  if  the  term  “Contractor’s  Proposed  System"  is  used  in  lieu  of  “Candidate  System 
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The  candidate  and  preferred  system(s)  having  been  defined  in  the  conceptual  phase, 
a sensitivity  analysis  is  performed  with  the  effectiveness  model.  This  analysis  will  indicate 
the  limiting  parameters  and  priorities  for  each  element  of  the  system  model,  which  are 
expressed  in  terms  of  technical  goals  or  requirements.  The  range  of  the  permissible  para- 
meters, properly  related  to  estimates  of  state-of-the-art  capabilities,  establishes  the  degree 
of  criticality  of  the  element. 

Along  with  the  sensitivity  analysis,  the  analysis  of  risk  and  uncertainty  performed 
during  the  conceptual  phase  should  be  updated,  extended,  and  carried  out  to  greater  levels 
of  detail. 

The  system(s)  definition  and  the  critical  system  effectiveness  parameters  are  incor- 
porated into  a request  for  proposal  (RFP),  which  is  transmitted  to  the  contractor(s)  as  a 
guide  for  proposed  approaches  to  validation.  The  system(s)  definition  provides  for  the 
initiation  of  validation,  the  equivalent  of  candidate  system  definition  in  the  conceptual 
phase.  In  addition  to  guidance  for  the  contractor(s),  the  definition  of  critical  system 
effectiveness  parameters  provides  the  criteria  for  evaluating  contractor  proposals. 

As  with  other  aspects  of  the  :system  effectiveness  discipline,  the  definition  of  criti- 
cal system  effectiveness  parameters  is  not  static.  The  process  of  refinement  started  in  the 
conceptual  phase  continues  in  validation.  As  a result  of  the  analysis  of  contractor  or 
laboratory  proposals,  the  system(s)  definition  is  refined,  and  the  critical  parameters  are 
better  defined  by  the  inputs  received  from  the  responses  to  the  RFP.  This  sharpening 
becomes  most  important  to  the  project  manager  during  latter  stages  of  validation  and 
during  full-scale  development. 

The  project  manager  may  exercise  little  control  over  the  validation  effort.  However, 
progress  reports  under  the  contracts  do  provide  definition  and  validating  data.  As  these 
data  are  received,  reiterative  exercise  of  the  system  effectiveness  model  provides  a signifi- 
cant measure  of  the  progress  being  realized. 

While  tne  critical  system  effectiveness  parameters  can  be  defined  initially  during 
the  early  conceptual  phases,  they  reach  much  greater  definition  in  the  latter  stages  of 
validation  during  the  analysis  effort.  They  provide  the  essential  framework  for  the  deci- 
sion to  enter  into  full-scale  development. 

The  definition  of  these  parameters  at  this  point  in  the  evolutionary  cycle  of  a sys- 
tem must  be  sharpened  to  the  point  where  the  project  manager  can  demonstrate  the  fol- 
lowing: 

• The  operational  goals  and  technical  goals  are  in  agreement. 

• The  technical,  and  hence  operational,  criteria  can  be  met. 

• The  financial  and  schedule  factors  are  credible. 

• The  development  risks  are  acceptable. 

• A definitive  full-scale  development  contract  can  be  entered  into  with  the  best- 
qualified  contractor. 

To  demonstrate  the  foregoing,  not  only  must  the  parameters  be  clearly  and  con- 
cisely defined,  but  they  must  be  quantitatively  interrelated.  This  requires  highly  structured 
system  models  in  terms  of  functional  block  diagrams  with  associated  characteristics  values 
and  a completely  structured  system  effectiveness/system  value  model  with  which  to  analyze 
and  evaluate  the  system  models.  The  former  is  an  output  of  validation  contractor  efforts. 
The  latter,  however,  is  largely  the  result  of  the  efforts  of  the  project  manager's  staff  or 


V-3 


research/analysis  contractor.  The  success  or  failure  of  validation  will  be  determined  by 
the  degree  of  completeness  of  the  model  and  the  degree  to  which  its  structuring  conforms 
to  the  real-world  situation. 

If  the  system  effectiveness/system  value  model  does  approximate  reality  success- 
fully, the  parameters  can  be  interrelated,  and  the  exercise  of  the  model  for  each  of  the 
competing  systems  produced  by  the  contractor(s)  provides  a framework  for  source  selec- 
tion and  demonstrates  the  validity  of  entering  into  full-scale  development,  continuing  with 
further  definition  or  advanced  development  effort,  or  abandoning  the  project. 

In  addition  to  its  use  as  a decision-making  tool,  the  model  also  serves  another 
function  during  this  period.  The  sharply  defined  critical  effectiveness  parameters  provide 
the  checklist  for  completing  the  specification  for  full-scale  development.  This  is  particu- 
larly important  in  that  one  of  the  principal  objectives  of  the  validation  process  is  to  assure 
that  a complete  and  unambiguous  specification  is  developed  for  the  full-scale  development 
effort. 


FULL-SCALE  DEVELOPMENT 


Through  the  process  of  validation,  the  project  manager  has  been  establishing  a 
frame  of  reference  to  define  the  system,  its  technical  goals  and  criteria,  and  tiie  measures 
by  which  its  effectiveness  in  terms  of  its  mission  life  costs  can  be  evaluated.  Having  es- 
tablished this  frame  of  reference,  he  must  now  address  himself  to  obtaining  assurance  of 
achieving  an  effective  system. 

During  full-scale  development,  the  weapon  system  including  all  the  items  necessary 
for  its  support  (training  equipment,  maintenance  equipment,  handbooks  for  operation  and 
maintenance,  etc)  is  designed,  fabricated,  and  tested.  The  intended  output  is  a “hardware 
model”  whose  effectiveness  has  been  proved  experimentally  together  with  tne  documenta- 
tion needed  for  inventory  use.  An  essential  activity  of  the  development  phase  is  test  and 
evaluation,  both  that  conducted  by  contractors  and  that  conducted  by  the  Service.  Docu- 
mentation of  the  full-scale  development  phase,  including  the  results  of  effectiveness  analysis, 
provides  the  basis  for  updating  the  DCP  and  convening  DSARC  leading  to  initiation  of 
production/deployment. 

The  ultimate  evaluation  of  the  full-scale  development  phase  occurs  during  the  test 
and  evaluation  of  the  developed  system.  If  the  system  model  and  system  effectiveness 
analytic  model  are  valid  and  adequately  defined,  the  system  should  meet  its  test  and  eval- 
uation successfully,  and  the  project  manager  will  have  been  successful. 

If  the  system  is  not  satisfactory,  the  models  have  yet  another  function.  The  data 
accumulated  during  T&E  should  be  inserted  into  the  models.  The  models  should  then  be 
exercised  and  the  results  analyzed  to  identify  problem  areas.  These  should  then  be  recorded 
and  made  available  to  other  project  managers  to  assist  them  in  avoiding  similar  errors.  At 
the  same  time,  a closed-loop  management  system  should  be  implemented  to  correct  the 
problems. 

If  the  project  is  to  be  continued,  whether  or  not  the  T&l  is  successful,  the  T&E 
data  arc  inserted  into  the  models  to  sharpen  further  the  definition  of  technical  goals  and 
criteria  and  to  validate  the  data  for  the  production  baseline  and  production  specification. 
Here,  again,  the  models  serve  to  guide  the  effort  and  to  assure  the  project  manager  that 
the  baseline  (specification)  is  complete  and  defined  as  sharply  as  the  aggregate  experience 
will  permit.  This  is  a necessary  exercise,  whether  or  not  the  R&D  contractor  is  also  the 
initial  production  contractor. 


INTEGRATED  LOGISTIC  SUPPORT  (ILS) 

VERIFICATION  AND  DEMONSTRATION 

The  program  requirements  for  Integrated  Logistic  Support  of  NAVELEX  procure- 
ments are  contained  in  MIL-STD-1369.  ILS  verification  and  demonstration  are  covered 
in  paragraph  4.8  and  appendix  C of  that  standard.  There  are  three  stages  for  verification 
and  demonstration  of  maintainability  and  integration  of  maintenance  resources.  The 
demonstration  and  validation  should  be  conducted  on  an  integrated  basis  consistent  with 
other  related  test  and  demonstration  requirements. 

STAGE  ONE 

Stage  one  should  be  progressively  implemented  in  accordance  with  MIL-STD-471 
during  breadboard  or  mock-up,  and  should  continue  through  evaluation  of  the  first  produc- 
tion end  article.  For  the  system/equipment,  with  its  subsystems  and  support  equipment, 
the  contractor  will  evaluate  accessibility,  simplicity,  equipment  size,  working  environment, 
maintenance  resource  requirements,  and  human  engineering  consideration,  and  determine 
whether  the  operational  requirements  can  be  met  without  exceeding  programmed  main- 
tenance resources. 

Within  30  days  after  the  verification  and  demonstration,  the  contractor  will  submit 
a report  of  the  verifications  to  the  Administrative  Contracting  Officer.  The  report  should 
include  all  pertinent  data  and  observations,  photographs  or  sketches  of  major  problem 
areas,  and  recommendations  for  corrective  action  as  required.  Subsequently,  maintaina- 
bility predictions  and  maintenance  resource  requirements  data  contained  in  the  Logistics 
Support  Analysis  (LSA)  should  be  updated. 


STAGE  TWO 

Stage  two  will  occur  concurrently  with  the  test  program  during  which  the  time 
and  achievement  of  the  end  article  maintainability  and  other  logistic  support  requirements 
will  be  verified.  The  verification  will  be  performed  on  a test  system  as  specified  in  the 
test  plan.  The  specific  time  phasing,  and  the  maintainability  and  other  logistic  support 
requirements  to  be  verified  and  demonstrated,  will  stipulated  by  the  contractor  agreed 
to  by  the  Naval  Electronic  Systems  Command,  and  included  in  the  maintainability  program 
plan.  The  demonstration  will  utilize  the  numbers  and  skill  levels  of  personnel  and  main- 
tenance resources  recommended  by  the  LSA  and  agreed  to  by  the  government  Publica- 
tions and  support  equipment  will  be  examined  for  adequacy,  compatibility,  and  capability 
to  support  the  established  maintenance  concept.  Maintainability  predictions  and  main- 
tenance resource  data  requirements  should  be  updated  during  this  stage  also. 


STAGE  THREE 

Demonstration  of  the  in-service  end  article  maintainability  characteristics  and  inte- 
gration of  maintenance  resources  will  be  accomplished  by  the  Naval  Electronic  Systems 
Command.  In-service  verification  will  be  accomplished  using  only  those  tools,  equipment. 
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data,  training,  personnel,  and  material  resources  which  have  been  programmed  and  pro- 
vided. The  in-service  demonstration  will  include  the  following  elements: 


1.  Preparation  and  Application.  The  contractor  must  prepare  the  ILS  Verifica- 
tion and  Demonstration  Plan  to  meet  the  criteria  for  demonstrating  whether 
or  not  the  system  or  equipment  support  requirements  have  been  attained. 

This  plan  must  be  agreeable  to  both  the  contractor  and  the  Navy.  The  de- 
monstration will  be  conducted  by  the  government  in  a typical  operational 
environment  with  contractor  participation  as  necessary  to  assure  mutual 
acceptability  of  test  data  and  the  analysis  thereof.  The  plan  must  provide 
for  assessment  of  system  maintainability  characteristics  as  well  as  support 
factors  related  to  item  downtime;  ie,  technical  manuals,  personnel,  tools, 
support  equipment,  maintenance  concept,  and  availability  and  adequacy  of 
required  spares  and  repair  parts. 

2.  Management.  The  ILS  verification  and  demonstration  will  be  managed  by 
the  government.  A Demonstration  Control  Board  will  be  established  to 
provide  on-site  management.  The  board  will  consist  of  three  to  five  govern- 
ment members,  one  of  whom  will  be  designated  as  the  Demonstration  Direc- 
tor, and  an  equitable  number  of  contractor  personnel  to  be  provided  at 
contractor  option  and  expense.  The  board  will  be  responsible  for  assuring 
that  maintainability,  maintenance,  and  support  data  are  collected  and  docu- 
mented in  accordance  with  established  Navy  policy  (or  an  approved  modifi- 
cation thereof  as  may  be  necessary);  determining  the  validity  of  data  report- 
ing; and  making  initial  determinations  as  to  whether  demonstration  objectives 
have  been  satisfied  and  contractual  requirements  have  been  met. 

3.  Demonstration  Location  and  Duration.  The  ILS  demonstration  will  preferably 
be  conducted  at  the  naval  activity  which  will  be  required  to  operate  and 
support  the  first  deliverable  production  system.  The  demonstration  will  com- 
mence approximately  6 months  after  delivery  of  the  system/equipment  to  the 
demonstration  site  and  may  continue  for  about  6 months.  This  will  allow 

for  operational  and  maintenance  familiarization  prior  to  the  demonstration 
and  will  provide  for  a demonstration  of  sufficient  scope  to  evaluate  maintenance 
and  support  requirements  for  a broad  operational  spectrum  of  the  system/ 
equipment. 

4.  Demonstration  Test  Team.  The  demonstration  test  team  will  consist  of  mem- 
bers of  the  demonstration  activity  and  the  Demonstration  Control  Board. 

5.  Demonstration  System  Equipment.  The  system/equipment  to  be  used  in  (he 
demonstration  will  be  that  regularly  assigned  to  the  activity  in  support  of  its 
assigned  mission.  No  attempt  will  be  made  to  segregate  a specific  group  of 
systems/equipments  for  demonstration  purposes.  All  assigned  systems/ 
equipments  will  be  used,  regardless  of  detailed  configuration,  provided  that 
such  systems/equipments  are  production  configured  and  delivered  to  the  Navy 
for  licet  training  and  operations.  No  specially  configured  test  system/equipment 
will  be  used  for  demonstration  purposes. 

6.  Maintenance  and  Support.  All  maintenance  performed  on  the  demonstration 
system/equipment  will  be  accomplished  by  demonstration  site  personnel  or 

by  personnel  attached  to  the  supporting  intermediate-level  maintenance  activity. 
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Organizational  and  intermediate-level  maintenance  will  be  performed  in 
accordance  with  the  approved/validated  technical  manuals  and  data  and  the 
support  resources  provided  by  the  system/equipment  1LS  program.  Depot- 
level  maintenance  will,  however,  be  performed  in  accordance  with  such  con- 
tractual requirements  as  may  be  applicable  during  the  period  in  which  the 
demonstration  is  performed.  No  organizational  or  intermediate  maintenance 
will  be  performed  by  contractor  personnel  during  the  demonstration  unless 
specifically  requested  by  the  demonstration  control  authority.  Similarly,  no 
contractor  advice  or  guidance  will  be  given  to  personnel  performing  mainten- 
ance unless  so  requested  by  the  control  authority. 

7.  Maintenance  Personnel.  The  composition  of  the  demonstration  test  team  and 
the  extent  of  training  of  the  personnel  involved  cannot  be  specified  except 
by  broad  parameters.  It  is  anticipated  that  the  activity  involved  will  be 
manned  with  a typical  mix  of  maintenance  personnel,  such  mix  to  follow  as 
closely  as  possible  the  maintenance  and  operating  factors  established  for  the 
system/equipment.  It  is  also  anticipated  that  a large  portion  of  the  organi- 
zational and  intermediate-level  maintenance  personnel  will  have  received 
either  factory  or  Navy  training.  Also,  time  will  be  allocated  for  on-the-job 
training,  as  required,  prior  to  commencement  of  the  demonstration. 

8.  Demonstration  Support  Material.  Initial  demonstration  site  surveys  for  veri- 
fication of  the  adequacy  of  logistic  support  status  will  be  conducted  by  the 
Demonstration  Control  Board  not  later  than  60  days  prior  to  the  scheduled 
commencement  of  the  demonstration.  Items  to  be  furnished  by  both  contrac- 
tor and  government,  based  on  the  approved  Support  Material  List  (SML),  will 
be  delivered  to  the  test  site  at  least  60  days  prior  to  commencement  of  the 
demonstration.  Thirty  days  prior  to  the  start  of  the  demonstration,  the 
Demonstration  Control  Board  will  survey  the  availability  and  serviceability  of 
support  material  and  initiate  action  through  the  program  management  office 
to  fill  shortages  and  replace  unserviceable  material.  The  Demonstration  Con- 
trol Board  will  also  make  recommendations  for  add-on  quantities  of  spares 
and  repair  parts.  Basis  for  such  recommendation  should  be  to  reduce  poten- 
tial program  delays.  Upon  completion  of  this  survey  and  all  possible  correc- 
tive actions,  a report  of  remaining  deficiencies  and  a recommendation  con- 
cerning program  start/delay  will  be  furnished  to  the  program  manager,  who 
will  decide  the  advisability  of  program  start  or  delay. 

9.  Maintenance  Data  Collection.  The  collection  of  accurate  data  and  the  analy- 
sis thereof  are  prerequisites  for  a successful  demonstration  program.  The 
data  must  have  a high  degree  of  accuracy  with  a bioad  base  for  analysis.  The 
data  system  utilized  to  collect  the  Dedicated  Maintenance  Man-Hours  (DMMH) 
requirements  will  be  the  Navy  3M  Data  System.  An  allowance  of  15%,  or 
that  percentage  agreed  to  by  the  government  and  the  contractor  during  con- 
tract negotiation,  for  PF&S  (personal,  fatigue,  and  supplementary)  time  will 
be  used  as  the  factor  to  convert  all  reported  DMMH  time  to  actual  DMMH 
required  time. 

10.  Analysis/EvaJuation  and  Report  Results.  Data  derived  from  the  Demonstra- 
tion Program  should  be  screened  thoroughly  for  accuracy,  classification  of 
data,  and  verification  of  mathematical  calculation.  Maintainability  measure- 
ments should  be  computed  as  specified  in  the  demonstration  plan.  A final 
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demonstration  report  must  be  prepared  by  the  demonstration  director  and 
submitted  to  the  government  program  manager  within  90  days  after  comple- 
tion of  the  demonstration. 

11.  Change  Incorporation.  In  the  event  a change  is  incorporated  during  the 

demonstration  period  that  affects  the  maintainability,  reliability,  or  support- 
ability  of  the  system/equipment,  the  contractor  may  request  a reevaluation  of 
the  applicable  demonstration  results  to  that  point  in  time,  provided  the  change 
is  incorporated  and  demonstrated  prior  to  preparation  and  submission  of  the 
demonstration  report. 

RELIABILITY  DEMONSTRATION 

All  test  programs  require  a test  plan.  One  part  of  the  reliability  test  plan  is  the 
demonstration  of  achieved  reliability.  Both  MIL-STD-785  and  MIL-R-22732  specify  the 
applicable  portions  of  MIL-STD-781  for  the  demonstration  test  procedure.  Additional 
requirements  for  the  reliability  demonstration  test  procedure  and  test  report  are  included 
in  MIL-R-22732.  The  reliability  test  plan,  including  the  demonstration  test  procedure, 
must  be  approved  by  the  procuring  activity  prior  to  initiation  of  the  tests.  Reliability 
tests  should  be  integrated  with  other  test  programs  to  avoid  costly  duplication  (eg,  per- 
formance testing,  flight  testing,  and  maintainability  demonstration). 

The  test  plan  establishes  ground  rules  for  conducting  tests  (including  GFE  impact), 
and  accept/reject  criteria  in  accordance  with  MIL-STD-781.  The  plan  also  includes  such 
items  as  demonstration  milestones,  confidence  or  risk  levels,  tradeoff  curves,  and  sample 
forms. 

The  test  level  may  be  in  accordance  with  MIL-STD-781,  MIL-R-22732,  or  an 
environmental  profile,  whichever  is  applicable. 

Along  with  the  reliability  demonstration  go  recorded  data  in  the  form  of  an  oper- 
ation log,  failure  reports,  and  failure  analysis  reports,  plus  a log  of  equipment  failures  and 
operating  time.  These  are  described  in*  MIL-R-22732  and  MIL-STD-781. 

MAINTAINABILITY  DEMONSTRATION 

To  prove  the  achievement  of  the  specified  maintainability  requirement,  a main- 
tainability demonstration  will  normally  be  accomplished  in  accordance  with  MIL-STD-471. 
The  contractor  must  prepare  and  submit  a demonstration  plan  and  report  to  the  procur- 
ing activity.  MIL-STD-471  establishes  procedures,  test  methods,  and  requirements  for 
verification,  demonstration,  and  evaluation  of  the  achievement  of  the  maintainability  re- 
quirement. The  formal  demonstration  is  conducted  in  an  operational  or  simulated  opera- 
tional environment  as  specified  in  the  contract.  The  demonstration  should  be  integrated 
with  other  testing  requirements  such  as  proof  of  design,  breadboard,  prototype,  environ- 
mental, production,  and  acceptance.  Data  from  the  demonstration  will  be  used  to  incre- 
mentally verify  the  achievement  of  maintainability  design  requirements  and  to  update  the 
parameter  values  from  the  maintainability  analyses  and  predictions. 
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HUMAN  ENGINEERING  TESTING 


GENERAL 

Tests  are  conducted  to  verify  that  designs  of  the  equipment,  software,  facilities, 
and  environment  meet  human  engineering  and  life  support  criteria  and  are  compatible 
with  the  overall  system  requirements.  The  test  and  evaluation  program  is  to  be  included 
in  the  Human  Engineering  Program  Plan.  Detailed  information  on  test  and  evaluation  can 
be  found  in  MIL-H-46855. 


STUDIES,  EXPERIMENTS,  AND  LABORATORY  TESTS 

The  contractor  shall  conduct  experiments,  laboratory  tests  including  dynamic  sim- 
ulation, and  studies  required  to  resolve  human  engineering  and  life  support  problems 
specific  to  the  system.  Human  engineering  and  life  support  problem  areas  shall  be  brought 
to  the  attention  of  the  procuring  activity  and  shall  include  the  estimated  effect  on  the  sys- 
tem if  the  problem  is  not  studied  and  resolved.  These  experiments,  laboratory  tests,  and 
studies  shall  be  accomplished  in  a timely  manner;  ie,  in  a manner  such  that  the  results 
may  be  incorporated  in  equipment  design.  The  performance  of  any  major  study  effort 
shall  require  approval  by  the  procuring  activity. 


MOCK-UPS  AND  MODELS 

At  the  earliest  practical  point  in  the  development  program,  and  well  before  fabri- 
cation of  system  prototypes,  full-scale,  three-dimensional  mock-ups  of  equipment  involving 
critical  human  performance  (such  as  an  aircrew  compartment,  maintenance  work  shelter, 
or  command  control  console)  shall  be  constructed.  The  proposed  Human  Engineering 
Program  Plan  shall  specify  mock-ups  requiring  procuring  activity  approval  and  modification 
to  reflect  changes.  The  workmanship  shall  be  no  more  elaborate  than  is  essential  to  deter- 
mine the  adequacy  of  size,  shape,  arrangement,  and  panel  content  of  the  equipment  for  use 
by  man.  The  most  inexpensive  materials  practical  shall  be  used  for  fabrication.  These 
mock-ups  and  models  shall  provide  a basis  for  resolving  access,  workspace,  and  related 
human  engineering  problems,  and  incorporating  these  solutions  into  system  design.  In 
those  design  areas  where  equipment  involves  critical  human  performance  and  where  human 
performance  measurements  are  necessary,  functional  mock-ups  shall  be  provided,  subject 
to  prior  approval  by  the  procuring  activity.  The  mock-ups  shall  be  available  for  inspection 
as  determined  by  the  procuring  activity.  Upon  approval  by  the  procuring  activity,  scale 
models  may  be  substituted  for  mock-ups.  Disposition  of  mock-ups  and  models,  after  they 
have  served  the  purposes  of  the  contract,  shall  be  as  directed  by  the  procuring  activity. 

DYNAMIC  SIMULATION 

Dynamic  simulation  techniques  shall  be  utilized  as  a human  engineering  design  tool 
when  necessary  for  the  detail  design  of  equipment  requiring  critical  human  performance. 
Consideration  shall  be  given  to  use  of  various  models  for  the  human  operator,  as  well  as 
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man-in-the-loop  simulation.  While  the  simulation  equipment  is  intended  for  use  as  a 
design  tool,  its  potential  relationship  to,  or  use  as,  training  equipment  shall  be  considered 
in  any  plan  for  dynamic  simulation. 

HUMAN  ENGINEERING  IN  TEST  AND  EVALUATION 

The  contractor  shall  establish  and  conduct  a test  and  evaluation  program  to:  (1) 
assure  fulfillment  of  applicable  requirements;  (2)  demonstrate  conformance  of  system, 
equipment,  and  facility  design  to  human  engineering  design  criteria;  (3)  confirm  compli- 
ance with  performance  requirements  where  man  is  a performance  determinant;  (4)  secure 
quantitative  measures  of  system  performance  which  are  a function  of  man-machine  inter- 
action; and  (5)  determine  whether  undesirable  design  or  procedural  features  have  been 
introduced.  (The  fact  that  these  functions  may  occur  at  various  stages  in  system  or 
equipment  development  shall  not  preclude  final  human  engineering  verification  of  the 
complete  system.  Both  operator  and  maintenance  tasks  shall  be  performed  as  described 
in  approved  test  plans  during  the  final  system  test.) 


PLANNING 

Human  engineering  testing  shall  be  incorporated  into  the  test  and  evaluation  pro- 
gram and  shall  be  integrated  into  engineering  design  tests,  contractor  demonstrations,  R&D 
acceptance  tests,  and  other  major  development  tests.  Compliance  with  human  engineering 
requirements  shall  be  tested  as  early  as  possible.  Human  engineering  findings  from  early 
testing  shall  be  used  in  planning  and  conducting  later  tests. 


IMPLEMENTATION 

The  human  engineering  test  and  evaluation  program,  contained  in  approved  test 
plans,  shall  be  implemented  by  the  contractor.  Test  documentation  (eg,  checklists,  data 
sheets,  questionnaires,  schedules,  operating  procedures,  and  test  procedures)  shall  be 
available  at  the  test  site.  Human  engineering  portions  of  all  tests  shall  include,  where 
applicable,  the  following: 

A simulation  (or  actual  conduct  where  possible)  of  mission  or  work  cycle. 

Tests  in  which  human  participation  is  critical  with  respect  to  speed,  accuracy, 
reliability,  or  cost. 

A representative  sample  of  noncritical  scheduled  and  unscheduled  maintenance 
tasks. 

Proposed  job  aids. 

Utilization  of  personnel  who  are  representative  of  the  range  of  the  intended  mili- 
tary user  population  in  terms  of  skills,  size,  and  strength:  and  wearing  suitable 
military  garments  and  equipment  appropriate  to  the  tasks  and  approved  by  the 
procuring  activity. 

Collection  of  task  performance  data. 
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Identification  of  discrepancies  between  required  and  obtained  task  performance. 
Criteria  for  the  acceptable  performance  of  the  tests. 


FAILURE  ANALYSIS 

All  failures  occurring  during,  or  as  a result  of,  test  and  evaluation  shall  be  subjected 
to  a human  engineering  review  to  differentiate  between  failures  due  to  equipment  alone, 
to  man-equipment  incompatibilities,  and  to  human  error.  The  procuring  activity  shall  be 
notified  of  design  deficiencies  which  contribute  to  human  error. 


SAFETY  TESTING 

The  system  safety  engineering  activities  are  required  to  establish  test  requirements 
and  ensure  that  safety  verification  of  design  and  data  are  included  in  the  engineering  test 
program. 

The  System  Safety  Program  Plan  (SSPP)  includes  the  manner  of  demonstrating 
quantitative  system  safety  requirements;  identification  of  special  safety  test  data;  and 
range,  flight,  and  operational  test  safety  programs. 

Tests  shall  be  proposed  in  the  SSPP  to  validate  the  safety  of  the  product,  including 
those  tests  already  specified  by  the  procuring  activity.  Safety  tests  shall  be  integrated 
into  appropriate  test  plans.  Where  complete  safety  testing  costs  would  be  prohibitive, 
partial  design  verification  of  safety  characteristics  or  procedures  may  be  demonstrated  by 
laboratory  test,  functional  mock-ups,  or  model  simulation,  when  approved  by  the  procur- 
ing activity.  Safety  tests  shall  be  performed  on  critical  devices  or  components  to  determine 
the  degree  of  hazard  or  margin  of  safety  of  design.  Induced  or  simulated  failures  will  be 
considered  for  demonstrating  the  failure  mode  of  critical  components.  The  detailed  test 
plans  for  all  tests  shall  be  reviewed  to  ensure  that; 

• Safety  is  adequately  demonstrated. 

• The  testing  will  be  carried  out  in  a safe  manner. 

• All  additional  hazards  introduced  by  testing  procedures,  instrumentation,  test 
hardware,  etc,  are  properly  identified  and  minimized. 

A system  safety  checklist  is  required  as  part  of  the  Contract  Data  Requirements 
List  (CDRL).  The  checklist  provides  assurance  that  all  required  and  identified  safety 
requirements  have  been  incorporated  in  the  design  and  hardware  and  verified  by  review, 
test,  or  other  method  approved  by  the  agency  concerned. 


ENVIRONMENTAL  TESTING 

ENVIRONMENTAL  CHARACTERISTICS 

Establishing  minimum  requirements  for  environmental  characteristics  is  especially 
significant  to  low-cost  new-equipment  development.  At  present,  all  equipments  intended 
for  shipboard  use  are  supposed  to  be  subjected  to  the  analysis  and  testing  rigors  of 


MIL-E-16400  for  shock,  vibration,  temperature,  and  humidity.  For  many  shipboard 
equipment  applications,  however,  MIL-E-16400  requirements  are  unnecessarily  rigorous 
and  consequently  result  in  unnecessarily  expensive  equipments.  For  this  reason,  the 
TELCAM  study  has  developed  vibration  and  temperature/relative-humidity  tests  less  severe 
than  tb^se  required  by  MIL-E-16400  — tests  which  will  provide  confidence  in  equipment 
endurance  under  usual  shipboard  environments  instead  of  survivability  under  abnormal, 
seldom-occurring  environmental  conditions.  (The  TELCAM  vibration  tests,  when  passed, 
also  serve  to  indicate  acceptable  performance  under  normal  shock  conditions.)  These 
TELCAM  tests  can  be  substituted  for  MIL-E-16400  tests  when  the  equipment  under 
development  (1)  is  noncritical  or  is  not  vital  to  survival  of  nor  essential  to  the  mission  of 
its  ship,  and  (2)  is  to  be  exposed  to  shock,  vibration,  temperature,  and  humidity  condi- 
tions found  inside  the  ship  forward  of  the  propeller  shaft(s).  This  will  help  to  reduce 
equipment  development  and  procurement  costs  and  yet  provide  an  equipment  which  will 
perform  adequately  under  all  normal  environmental  conditions.  Appendix  A discusses 
the  TELCAM  approach  to  environmental  requirements  and  presents  the  test  procedures 
to  be  used  when  specifying  the  quality  assurance  provisions  for  required  TELCAM  environ- 
mental characteristics. 


ESTABLISHING  SHOCK  AND  VIBRATION  CHARACTERISTICS 


To  establish  the  minimum  satisfactory  environmental  characteristics  for  shock  and 
vibration,  refer  to  figure  V-l  and  the  following  options. 


Option  1.  For  noncritical  equipments  to  be  used  only  in  Area  III  - the  least 
vibrationally  severe  area  - of  the  ship(s)  on  which  the  equipments  are  to  be  installed, 
establish  vibration  requirements  which  the  equipment  must  meet  after  being  subjected  to 
the  test  presented  in  appendix  A,  the  numerical  values  of  which  are  given  in  table  A-l  of 
the  appendix.  The  test  is  a broad  one,  covering  the  Area  III  vibration  characteristics  of 
all  ships.  On  some  ships,  however,  these  characteristics  are  less  severe  than  on  others. 
When  the  equipment  is  to  be  used  exclusively  in  a ship  or  ships  having  less  severe  vibra- 
tion characteristics  than  those  for  which  the  test  is  intended,  the  test  (and  requirements) 
can  be  made  less  constraining  by  substituting  the  specific  ship  characteristics,  presented  in 
figures  A-l  through  A-8  of  appendix  A,  in  place  of  the  values  in  table  A-l. 

Option  2.  For  noncritical  equipments  to  be  used  primarily  in  Area  III  but  with 
some  to  be  located  in  Area  I or  II,  establish  the  option  1 vibration  test  requirements,  and, 
in  addition,  for  the  percentage  of  equipments  in  Areas  I and  II,  establish  separate  packag- 
ing and  testing  requirements  that  will  allow  these  equipments  to  meet  MIL-STD-167  (see 
fig  A-l  in  appendix  A).  When  the  costs  for  extra  packaging  for  the  required  number  of 
equipments  equals  or  exceeds  80%  of  the  cost  that  would  be  required  to  extra-package 
all  the  equipments,  then  establish  shock  and  vibration  requirements  for  all  equipments  as 
per  the  following  option  3. 

Option  3.  For  noncritical  equipments  to  be  used  entirely  or  almost  entirely  in 
Area  I or  II,  and  for  all  critical  equipments,  establish  the  level  and  testing  requirements 
of  MIL-STD-167  for  vibration  and  of  MIL-S-901  for  shock.  Instead  of  making  general 
reference  to  these  documents,  pinpoint  only  those  requirements  necessary  for  survival  by 
citing  specific  paragraphs. 


ESTABLISHING  TEMPERATURE  AND  HUMIDITY  CHARACTERISTICS 

To  establish  the  minimum  satisfactory  environmental  characteristics  for  tempera- 
ture and  humidity,  refer  to  figure  V-2  and  the  following: 

1.  For  noncritical  equipments  to  be  located  inside  a ship  (Areas  I and  III,  fig 
V-l)  — ie,  for  those  class  4 (MIL-E-16400)  equipments  which  are  not  exposed 
directly  to  weather  — establish  the  temperature-humidity  requirements  pro- 
filed by  figure  V-2  and  test  by  making  five  complete  cycles  around  the  trape- 
zoid of  the  figure.  Each  test  cycle  starts  and  finishes  at  95°F  and  95%  rela- 
tive humidity,  with  each  of  the  test-condition  points  being  maintained  for  5 
hours  and  with  1 hour  allowed  for  transition  between  the  points  - a 24-hour 
period. 

2.  For  equipments  to  be  located  in  more  exposed  areas  — ie,  for  class  1,  2,  and 
3 (MIL-E-16400)  equipments  - establish  the  temperature  and  humidity 
requirements  imposed  by  MIL-E-16400  for  the  corresponding  equipment 
class  (refer  to  paragraphs  3.3.5  and  4.8.3  of  MIL-E-16400G). 
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Figure  V-2.  TELCAM  temperature-humidity  test. 
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ESTABLISHING  OTHER  ENVIRONMENTAL  CHARACTERISTICS. 

Other  environmental  constraints  concern  noise,  enclosures,  inclination,  wind  speed, 
EMI,  etc,  as  covered  by  MIL-E-16400.  They  may  or  may  not  have  to  be  characterized  and 
tested,  depending  on  the  operational  requirements  of  the  equipment  characteristics  being 
established.  If  they  do,  the  requirements  of  MIL-E-16400  should  be  cited.  Refer  to 
figure  V-3. 

To  be  especially  noted  is  that  airborne  and  structurebome  noise  and  enclosure 
requirements  and  tests  should  be  applied  to  all  sheltered  equipments.  Inclination  require- 
ments and  tests  should  be  applied  only  to  equipments  whose  proper  operation  depends  on 
fluid  levels,  position-sensitive  switches,  or  heat  pipes,  or  which  are  otherwise  position  sensi- 
tive, since  this  constraint/test  has  a motion  to  which  most  equipments  are  insensitive. 


ENVIRONMENTAL  REFERENCES 

1 . Naval  Air  Systems  Command 
Military  Standard  M1L-STD-810C, 

Environmental  Test  Methods,  10  March  1975 

2.  Naval  Ship  Engineering  Center 
Military  Standard  MIL-STD-1399A, 

Interface  Standard  for  Shipboard  Systems.  20  Dec  72 
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3.  Naval  Electronic  Systems  Command 

Military  Specification  MIL-E-2036C,  Enclosure  for 

Electric  and  Electronic  Equipment,  Naval  Shipboard.  1 5 Mar  63 


4.  Naval  Ordnance  Systems  Command 
Military  Specification  M1L-C-45662A, 

Calibration  System  Requirements,  9 Feb  62 

5.  Naval  Ship  Engineering  Center 

Military  Standard  MIL-STD-740B,  Airborne  and 
Structureborne  Noise  Measurements  and  Acceptance 
Criteria  of  Shipboard  Equipment,  13  Jan  65 

6.  Naval  Ship  Engineering  Center 

Military  Standard  MIL-STD-108E,  Definitions  of  and 
Basic  Requirements  tor  Enclosures  for  Electric  and 
Electronic  Equipment,  4 Aug  66 

7.  Naval  Air  Systems  Command 

Military  Specification  M1L-E-6051D,  Electromagnetic 
Compatibility  Requirements,  Systems,  7 Sep  67 

8.  Air  Force  Logistics  Command 

Military  Specification  MIL-R-9673B,  Radiation  Limits, 

Microwave  and  X-Radiation  Generated  by  Ground  Electronic- 
Equipment  (as  Related  to  Personnel  Safety),  15  Sep  61 

9.  Naval  Ordnance  Systems  Command 

Military  Standard  MIL-STD-1385,  Preclusion  of  Ordnance 

Hazards  in  Electromagnetic  Fields,  General  Requirements  for,  6 Apr  72 

10.  Naval  Air  Systems  Command 

Military  Standard  M1L-STD-704A,  Electric  Power, 

Aircraft,  Characteristics  and  Utilization  of,  9 Aug  66 

1 1 . Naval  Ship  Engineering  Center 

Military  Standard  M1L-STD-463,  Definitions  and  Systems  of 
Units,  Electromagnetic  interference  Technology,  9 Jun  66 

12.  Naval  Facilities  Engineering  Command 
Military  Standard  MIL-STD-633C, 

Mobil  Electric  Power  Engine  Generator  Standard 
Family  Characteristics  Data  Sheet,  29  Oct  71 

13.  Naval  Air  Systems  Command 
Military  Standard  M1L-STD-831, 

Test  Reports,  Preparation  of.  28  Aug  63 

14.  Naval  Electronic  Systems  Command  Military 

Standard  MIL-STD-461  A,  Electromagnetic  Interference  .« 

Characteristics,  Requirements  for  Equipment.  1 Aug  (>H 

15.  Naval  Electronic  Systems  Command  Military  Standard 
MIL-STD^f62.  Electromagnetic  Interference 
Characteristics,  Measurement  of.  31  Jul  67 
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16.  Naval  Ship  Engineering  Center  Military  Standard 

MIL  STD-469,  Radar  Engineering  Design  Requirements, 
Electromagnetic  Compatibility,  1 Dec  66 

17.  Naval  Snip  Engineering  Center  Military 
Specification  M1L-S-901C,  Shock  Tests,  H.l. 
(High-Impact),  Shipboard  Machinery,  Equipment 
and  Systems,  Requirements  for,  5 Sep  63 


18.  Naval  Ship  Engineering  Center  Military  Standard 
MIL-STD-167/1 , Mechanical  Vibrations  of 
Shipboard  Equipment,  1 May  74 

19.  Vibration  Test  Data  Reports: 

Source  Title  Rprt  No 


Boston  Naval 
Shipyard: 


Longbeach  Naval 
Shipyard: 


.1 . “Collection  of 
Reports  on 
Vibration  Surveys" 

2.  same 

3.  same 

4.  same 

AD  458905 

5.  same 

AD  479764 

6.  same 

AD  813701 

7.  same 

1 . “Collection  of 
Reports  of 
Vibration  Surveys” 

2.  “Collection  of 

Vibration  Reports" 
3.  “Collection  of 

Reports  of  Vibra- 
tion Surveys” 

4.  same 

AD  460569 

5.  same 

AD  428086 

6.  same 

AD  827529 

7.  same 

AD  847891 

8.  same 

- 

9.  same 

— 

10.  “USS  Iwo 

Jima  (LPH  2) 
Underway  Vibra- 
tion Survey” 

Date 


1961 


1962 

1963 
!%4 

1965 

1 966 
1969 


1961 


1962 

1963 


1964 

1965 

1967 

1968 

1 969 

1970 
1972 
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Source 


Title 


Rprt  No 


Date 


Norfolk  Naval 
Shipyard: 


Pearl  Harbor 
Naval  Shipyard: 


Philadelphia  Naval 
Shipyard: 


Portsmouth  Naval 
Shipyard: 


Puget  Sound 
Naval  Shipyard: 


1.  “Vibration  - 1961 

Survey  Reprots” 

2.  same  — 1962 

3.  same  — 1963 

4.  same  — 1964 

5.  same  AD  482035  1965 

6.  same  AD  818880  1966 

7.  same  AD  834603  1967 

8.  same  — 1968-1969 

9.  same  — 1972 

1.  “Vibration  — 1961 

and  Noise  Survey 

Reports” 

2.  same  — 1 963 

1 . “Collection  of  — 1961 

Letter  Reports  of 

Local  Vibration 
Surveys” 

2.  same  — 1962 

3.  same  - 1963 

4.  “Collection  of  AD  809229  1964 

Vibration  Surveys” 

5.  “Collection  of  — 1969 

Vibration  Surveys 

and  Letter  Reports” 

6.  “Collection  of  - 1970 

Vibration  Surveys” 

1.  “Reports  on  — 1962 

Vibration  Surveys” 

1 . “Collection  of  - 1961 

Reports  on  Vibra- 
tion Surveys” 

2.  same  - 1962 

3.  same  — 1963 

4.  same  AD  463274  1964 

5 same  AD  481976  1965 

6.  same  AD  809213  1966 

7.  same  AD  834332  1967 
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Source 

San  Francisco  Bay 
Naval  Shipyard: 

Title 

Rprt  No 

Date 

1.  "Shipboard 
Vibration  Survey 
of  1962” 

1962 

2.  "Collection  of 
Vibration  Surveys 
for  1963” 

1963 

3.  “Shipboard  and 
Vibration  Memos 
and  Surveys  for 
1964” 

AD  466652 

1964 

4.  “Collection  of 
Vibration  Surveys 
for  1965” 

1965 

Charleston 
Naval  Shipyard: 

5.  same  . . . ”1966” 

AD  8 15849 

1966 

Naval  Electronics 

Laboratory 

Center: 

1 . "Informal  and 
Letter  Reports  on 
Vibration  Prob- 
lems for  1 964” 

AD  460923 

1964 

1 . D.  Peterson, 
“Summary  of 
NELC”  Recorded 
Shipboard  Vibra- 
tion Data 

1701 

2 April  1970 

Naval  Ship 
Research  and 
Development 
Center,  Washington, 
D.C. 

2.  R.H.  Chalmers 
and  D.L.  Peterson 
“Environmental 
Studies  Aboard 
U.S.  Navy  Vessels 
in  the  South 
China  and 
Caribbean  Seas” 

1577 

1 6 August  1 968 

1.  V.S.  Hardy,” 
“Surface  Ship 
Vibration” 

NSRDC  2338 

June  1967 

2.H.F.  Alma  and  NSRDC  2338A 
N.W.  Huzil,  “Surface 
Ship  Vibration” 

Sept.  1970 
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20.  NELC  Memo  Z269,  Ser  4700-M205-73  dated  27  December  1973,  from 
TA  Danielson  to  DA  Peterson  (Subj:  Supply  Line  Voltage  and 
Frequency  Test  Results  on  Century  Data  Co.  “Floppy  Disk  Memory 
Unit”  Ser  No  512) 

21 . NELC  Letter  Z269,  Ser  4700-82  dated  5 December  1973,  from 
Commander  NELC  to  Commander  NUC  (Subj : Results  of  exploratory 
vibration  test  of  CALCOMP  Model  565  Plotter) 


INTERFACE  REQUIREMENTS  VALIDATION 

The  need  for  interface  standards  for  shipboard  systems  has  become  apparent  as  ships 
and  their  systems/equipment  have  grown  more  complex  and  as  discrete  activities  (SYSCOMs, 
PMs,  Contractors,  etc)  involved  in  ship/equipment  design  have  proliferated.  A Shipboard 
Interface  Standards  Program  has  been  established  to  meet  this  need.  Shipboard  interfaces, 
when  considered  in  their  totality,  confront  the  systems/design  engineer  with  complex  and 
intricate  problems.  Solution  of  these  problems  is  facilitated  by  defining  particular  selected 
interfaces  and  the  constraints  they  impose  on  equipment  which  may  be  involved  at  them. 
MIL-STD-1399  is  structured  to  provide  these  definitions.  This  standard,  and  its  support- 
ing sections,  defines  the  constraints  on  systems/equipment  design  imposed  by  the  estab- 
lished characteristics  of  the  shipboard  environment.  It  will: 

• Specify  constraints  for  systems/equipment  design  imposed  by  the  shipboard 
environment 

• Provide  for  early  dialog  among  personnel  concerned 

• Assist  in  achieving  more  effective  ship  configuration  management 

• Contribute  to  cost-effective  and  integrated  ships  by  promoting  interface  com- 
patibility 

• Ultimately  achieve  interface  compatibility  of  installed  systems/equipment  with 
the  shipboard  environment 

MIL-STD-1399  applies  to  all  activities  involved  in  ship/systems/equipment  design, 
production,  and  installation  and  requires  that  the  interfacing  aspects  of  ship/systems/ 
equipment  be  given  priority  consideration  by  such  activities.  The  specific  interface  charac- 
teristics and  constraints  established  in  the  various  sections  of  this  standard  are  mandatory 
and  shall  be  adhered  to  by  SYSCOMs,  PMs,  contractors,  and  all  others  engaged  in  any 
aspect  of  total  shipboard  design,  including  system/equipment  design,  production,  and 
installation.  It  is  incumbent  upon  interested  activities,  in  consonance  with  the  objectives 
ot  the  Interface  Standards  Program,  to  establish  a dialog  in  a timely  manner  which  will 
bring  into  focus  and  resolve  any  interface  problems  which  may  require  attention  in  areas 
not  yet  covered  by  this  standard.  It  is  essential  that  interface  requirements  be  carefully 
considered  by  all  naval  activities  and  contractors  involved  in  ship  construction/moderniza- 
tion/conversion throughout  the  entire  ship  life  cycle.  It  is  also  mandatory  that  Principal 
Development  Activities  (PDAs)  invoke  this  standard  in  the  Development  Plans  (DPs)  and 
procurement  specifications  for  new  systems/equipment  destined  for  shipboard  installation. 
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REFERENCE 


Naval  Ship  Engineering  Center  Military  Standard,  M1L-STD-1 399 A,  Interface 
Standard  for  Shipboard  Systems,  20  Dec  72 

DEFICIENCY  ANALYSIS 

During  the  validation  testing,  analyze  any  failures  or  deficiencies  as  to  their  impact 
on  the  program.  Quick  fixes  may  have  been  incorporated  during  testing,  and  further  mod- 
ification may  be  required  for  service  use.  The  extent  of  the  deficiencies,  failures,  or 
required  modifications  will  determine  the  course  of  action  at  this  point.  The  next  step  can 
range  from  obtaining  service  approval  to  requesting  more  funds  for  modification  or  further 
development,  or  even  to  canceling  the  program. 

In  the  event  that  modification  is  called  for,  the  program  plan  will  (may)  also 
require  modification.  This  includes  new  schedules,  readjusting  fund  allocations  (new  or 
existing  funds),  and  most  of  the  other  steps  covered  in  chapter  III,  Program  Planning. 

The  technical  approach  may  also  need  modification.  Refer  to  chapter  IV,  Concep- 
tual Phase.  This  will  ultimately  lead  to  updating  the  system  specification.  Major  changes 
in  the  technical  approach  will  require  new  validation  tests.  Refer  to  the  introduction  to 
this  section.  Those  elements  of  the  system  that  do  not  change  may  still  be  covered  by 
the  original  validation,  depending  on  the  impact  of  the  change. 
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VI.  TRANSITION  TO  PRODUCTION 


The  transition  to  production  phase  is  simply  the  translation  of  a viable 
system  concept  into  a form  which  can  be  efficiently  produced  in  the 
quantities  required  for  service  use.  Off-the-shelf  systems  or  readily  modi- 
fied systems  involve  a straightforward  transition;  developments  may  cause 
a very  complex  transition.  This  phase  is  almost  entirely  dictated  by  the 
decisions  made  in  prior  phases;  therefore,  planning  incorporating  the  con- 
siderations discussed  in  this  chapter  must  be  accomplished  much  earlier  in 
the  project  cycle,  usually  in  the  conceptual  phase. 


Once  the  technical  approach  has  been  validated,  the  program  is  ready  to  complete 
the  acquisition  cycle.  At  this  transition  point,  the  decisions  to  build,  buy,  or  modify 
equipments  to  suit  the  requirements  must  be  finalized. 


Alternative 

Phase 

Major  Phase 

Build 

Full-Scale  Development  (EDM) 

Engineering  Qualification 

Product  Development 

Modify 

Modifications 

Buy 

TECHEVAL/OPEVAL 

Service  Approval 

Qualification 

Production 

Deployment 

Initiation 

While  the  actions  required  in  each  phase  may  differ  among  the  alternatives,  it  is  possible  to 
assemble  a system  with  equipments  acquired  through  each  alternative;  in  such  a case,  system 
integration  is  always  a major  issue  in  the  qualification  phase. 

There  are  two  major  milestones  in  the  transition  to  production  — Approval  for 
Service  Use  (ASU)  and  Initial  Operational  Capability  (IOC).  Refer  to  chapter  VII,  Approval 
for  Service  Use.  for  the  requirements  for  meeting  the  ASU  milestone.  IOC  is  the  date  that 
the  deployment  phase  is  initiated.  Another  milestone.  Final  Operational  Capability  (FOC). 
delineates  the  completion  date  ol  the  deployment  phase.  IOC  is  extremely  important 
because  the  equipment  logistics  support  must  be  initiated  by  IOC.  In  the  many  instances 
in  which  the  support  has  not  been  initiated  in  consonance  with  the  equipment,  the  result 
has  been  a degrading  of  the  equipment,  frequently  leading  to  permanently  unacceptable 
performance.  I(X  may  be  a firm  or  a flexible  milestone.  It  is  a firm  milestone  when  the 
equipment  must  be  available  on  a specific  date  such  as  when  it  is  GFE  to  a larger  contrac- 
tual effort  (such  as  new-ship  construction).  Flexible  IOC  exists  when  the  equipment 
delivery  is  required  only  to  meet  mission  needs.  The  firm  milestone  may  be  slipped  in 
actuality;  however,  the  IOC  must  be  assumed  to  be  fixed  for  planning  purposes.  The  flex- 
ible milestone  may  actually  be  quite  inflexible  because  of  ship  availability  schedules  for 


installation  and  for  political  reasons.  The  IOC  should  be  established  at  the  initiation  of 
the  program  and  guide  the  program  planning  actions  accordingly;  therefore,  the  flexibility 
of  the  IOC  must  be  known  in  order  to  assess  schedule  risks. 

MAJOR  DECISIONS 

Whichever  acquisition  alternative  is  pursued,  there  are  five  major  issues  which  must 
be  resolved  within  the  program  objectives  and  constraints.  These  are: 

• System  partitioning/integration 

• Design  ownership 

• Specification  type 

• Level  of  repair  performed  at  the  field  level 

• Standardization 

Each  of  these  issues  may  be  politically  volatile  depending  upon  the  sponsoring  activity 
and  the  time  within  the  acquisition  cycle  when  a decision  is  made.  In  general,  the  earlier 
within  a program  that  each  issue  is  addressed,  the  better  the  prospects  for  a smooth  transi- 
tion. This  is  true  not  only  from  the  standpoint  of  being  able  to  preplan,  but  from  the 
standpoint  of  political  risk  as  well.  An  early  decision  point  allows  time  to  budget  properly 
and  to  bring  the  parties  involved  in  the  acquisition  to  an  understanding  of  the  acquisition 
plans;  both  elements  serve  to  mollify  opposing  views. 

SYSTEM  PARTITIONING/INTEGRATION 

The  first  and  most  complex  issue  involves  the  system  partitioning  and  the  subse- 
quent integration  of  the  pieces  into  the  system.  The  degree  of  partitioning  possible  lor 
a system  is  directly  proportional  to  the  complexity  of  the  system;  a very  complex  system 
will  be  partitioned  within  a hierarchy  of  levels  of  varying  degrees  of  complexity  as  illus- 
trated below: 

Level 

1 

2 

3 

4 

5 

6 

7 

8 

9 


Definition 

Piece  part 
Module 

Subassembly  or  Shop  Replaceable  Assembly  (SR A) 
Assembly 

Unit  or  Weapon  Replaceable  Assembly  (WRA) 

Group 

Set 

Subsystem 

System 
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The  definitions  above  conform  to  M1L-STD-280  except  for  Module,  WRA,  and  SRA. 

Module  was  added  in  recognition  of  the  great  increases  in  system  complexity  within  re- 
cent years;  additional  levels  could  be  accommodated  in  a like  manner.  WRA  and  SRA 
are  defined  in  MIL-STD-1390.  Obviously,  less-complex  equipments  have  fewer  levels  of 
complexity.  A single-function  simple  equipment  might  have  only  two  levels  - equipment 
and  piece  part.  See  figure  VI-1.  The  necessity  of  breaking  down  the  system  for  cost 
estimating,  scheduling,  work  package  formulation,  etc,  is  evident  in  the  utility  of  the  Work 
Breakdown  Structure  (WBS)  (see  chapter  III,  Program  Planning).  Partitioning  is  also  an 
important  step  in  resolving  the  other  four  major  issues.  The  partitioning  phase  normally 
takes  place  during  concept  formulation  (see  chapter  IV,  Conceptual  Phase). 

Once  partitions  are  established,  the  system  integration  problem  begins.  The  pri- 
mary issue  to  be  resolved  regarding  system  integration  is  where  the  responsibility  tor  the 
tasks  lies  — which  commands/activities  within  the  government  and  which  industrial  com- 
panies have  major  and  subordinate  roles.  The  Systems  Engineer,  responsible  as  the  system 
integrator,  must  do  the  following: 

1.  Transform  an  operational  need  into  a description  of  system  performance 
parameters  and  a system  configuration  throi  th  the  use  of  an  iterative  process 
of  definition,  synthesis,  analysis,  design,  test,  and  evaluation. 

2.  Integrate  related  technical  parameters  and  assu.e  compatibility  of  all  physical, 
functional,  and  program  interfaces  in  a manner  which  optimizes  the  total 
system  definition  and  design. 

3.  Integrate  reliability,  maintainability,  safety,  human,  and  other  such  factors 
into  the  total  engineering  effort. 

System  engineering  effort  includes,  for  example,  system  definitization,  overall  system  de- 
sign, design  integrity  analysis,  system  optimization,  cost/effectiveness  analysis,  weight  and 
balance  analysis,  and  intrasystem  and  intersystem  compatibility  analysis.  It  also  includes 
reliability,  maintainability,  safety  and  survivability  program  requirements,  human  engineer- 
ing and  manpower  factors  program,  preparation  of  equipment  and  component  performance 
specifications,  security  requirements,  logistics  support  integration,  and  design  of  test  and 
demonstration  plans.  , 

if  the  operational  needs  are  to  be  met  by  the  system  design,  it  is  virtually  impossi- 
ble for  the  government  to  delegate  these  ultimate  system  engineering  responsibilities  to 
industry  since  a detailed  knowledge  of  the  operational  need  is  required;  however,  these 
responsibilities  normally  are  delegated  to  industry  for  portions  ot  the  system  below  a given 
level  of  complexity.  Attempts  to  contract  the  ultimate  system  engineering  tasks  will 
nullify  the  government’s  control  of  the  technical  effort  resulting  in  a high  degree  of  risk 
in  obtaining  a product  acceptable  to  the  user.  Even  when  the  operational  need  can  be 
well  described  to  the  contractor,  the  government  will  not  be  in  a position  to  determine 
optimal  tradeoffs.  When  the  need  is  not  well  defined,  only  the  government  is  in  a posi- 
tion to  estimate  the  elements  needed  to  fill  in  the  holes.  System  integration  task  respon- 
sibilities should  be  delegated  and  divided  along  the  same  partition  boundaries  as  the  system 
engineering  tasks.  The  division  of  responsibilities  should  be  clearly  established,  consistent 
with  the  determinations  made  for  the  other  major  issues,  and  always  implying  total  industry 
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QUANTITY 


INTERMEDIATE  = LARGE  WHEN  EACH  ITEM  COST  IS  (RELATIVELY)  HIGH 
INTERMEDIATE  = SMALL  WHEN  EACH  ITEM  COST  IS  (RELATIVELY!  LOW 


accountability  to  the  government  for  the  assigned  tasks  (ie,  the  government  should  not  assume 
responsibility  for  an  item  which  is  to  be  integrated  into  an  item  for  which  industry  is  respon- 
sible unless  the  partition  interfaces  are  fully  specified  and  validated).  This  latter  point  is  the 
most  controversial,  risky,  and  difficult-to-implement  aspect  of  system  integration,  and  it  is 
the  most  important.  Figure  VI-2  illustrates  a reasonable  resolution  of  this  issue. 

One  of  the  major  decisions  to  be  made  is  whether  to  task  the  physical  assembly 
of  the  final  system  to  industry  or  to  use  in-house  resources.  If  a single  contractor  or 
prime  contractor  is  being  utilized,  that  contractor  should  be  tasked;  however,  if  the  system 
consists  largely  of  existing  equipments  from  diverse  sources,  in-house  resources  should  be 
utilized.  In  either  case,  the  government  must  develop  the  documentation  within  its  area 
of  direct  integration  responsibilities. 


DESIGN  OWNERSHIP 

The  issue  of  design  ownership  is  extremely  controversial  and  politically  sensitive, 
and  therefore  must  be  resolved  with  a well  documented  and  well  justified  decision.  The 
decision  to  be  made  is  whether  the  system  design  should  be  government  owned  or  con- 
tractor owned;  the  controversy  arises  because  it  is  generally  assumed  that  design  ownership 
must  be  either  one  or  the  other.  Actually,  the  requirements  for  the  government  to  own 
the  design  seldom  dictate  an  exclusive  decision,  and  practical  solutions  can  be  formulated 
whereby  the  government  owns  part  of  the  design  and  the  contractor  owns  the  rest.  The 
government  ownership  requirements  are  for  designs  meeting  one  or  more  of  the  following 
conditions: 

1.  The  design  is  based  upon  in-house  expertise  which  far  exceeds  industrial 
expertise. 

2.  Design  features  closely  interact  with  service  doctrine  and  directly  affect  the 
user’s  performance  of  duty. 

3.  Configuration  control  must  be  maintained  for  repair  and  standardization 
purposes. 

4.  There  is  essentially  no  commercial  market,  and  government  requirements 
are  recurring  but  insufficient  to  support  continuous  production  or  multi- 
source production. 

The  above  conditions  dictate  that  the  government  should  own  at  least  a portion  of  all 
designs.  However,  condition  (2)  features  can  often  be  specified  in  a functional  specifica- 
tion without  regard  to  the  design  implementing  those  features,  and  condition  (3)  applies 
only  to  the  level  of  standardization  which  must  be  maintained.  Therefore,  under  many 
conditions  the  government  is  only  interested  in  owning  a portion  of  the  design  - usually 
the  functional  design  down  to  some  level  of  complexity  supplemented  by  the  detailed 
(fabrication)  design  in  limited  portions  of  the  system  meeting  one  of  the  above  conditions 
(usually  (1)  or  (3)). 

Detailed  design  ownership  is  always  expensive  to  acquire.  Furthermore,  detailed 
designs  dictate  significantly  more  government  responsibility  in  the  development,  qualifica- 
tion. and  use  of  the  design.  Specifically,  the  government  must  procure  more  documenta- 
tion and  assume  the  responsibility  that  the  documentation  is  accurate  when  it  is  utilized 
in  production.  Also,  proprietary  parts  and  processes  which  are  common  to  sophisticated 
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technologies  must  be  excluded  from  the  design  or  else  the  government  must  obtain  license 
and  documentation  for  the  part  or  process.  This  negates  the  advantages  of  the  proprietary 
feature  for  the  contractor  and  therefore  is  hard  to  obtain.  The  other  option  for  the  gov- 
ernment is  to  be  satisfied  with  a permanent  sole-source  situation  which  will  probably 
negate  any  advantage  to  design  ownership.  More  aspects  of  detailed  design  ownership  are 
discussed  as  part  of  the  tradeoff  between  functional  and  fabrication  specifications  (see 
SPECIFICATION  TYPE,  below).  In  general,  detailed  design  ownership  requires  close 
monitoring  of  the  design  development  by  technically  knowledgeable  government  personnel, 
validation  of  the  developed  design  to  prove  conformance  to  the  functional  design  require- 
ments and  to  demonstrate  the  reproducibility  of  the  design,  maintenance  of  an  up-to-date 
configuration,  and  exclusion  of  proprietary  parts  and  processes.  The  validation  of  the 
design  is  extremely  important  and  consists  of  an  unbiased  analysis  of  the  allowable  toler- 
ances of  each  part  to  ensure  conformance  to  the  functional  design  under  worst-case  con- 
ditions. This  may  consist  of  a “simple”  study  for  straightforward  designs.  However, 
complex  designs  or  designs  utilizing  complex  processes  may  require  the  building  of  hard- 
ware by  a government  facility  or  by  another  contractor  to  proof  the  design  data  package. 

As  the  complexity  of  a design  increases,  the  risks  associated  with  not  validating 
the  design  data  package  increase  rapidly.  These  risks  manifest  themselves  as  a probability 
that  the  design  will  not  perform  the  required  functions  or  that  the  design  will  fail  to  fit 
together  or  that  the  required  production  processes  will  not  be  reproducible.  In  any  case, 
the  government  as  the  design  owner  must  bear  the  cost  of  making  whatever  changes  are 
necessary  to  produce  a correct  design.  Changes  under  these  conditions  can  easily  double 
the  costs  of  the  design  development.  If  the  development  costs  are  a significant  portion 
ot  the  total  program  costs  (over  5-10*70.  a design  validation  phase  should  be  considered 
mandatory.  As  an  additional  note,  changes  which  occur  during  design  validation  must 
themselves  be  proofed  by  unbiased  analysis. 

The  other  key  elements  to  acquiring  detailed  design  ownership  are  technically 
knowledgeable  design  management  and  good  configuration  control;  this  is  true  whether 
the  government  or  the  contractor  will  own  the  design.  If  the  government  will  assume 
ownership,  these  elements  must  reside  within  the  government.  If  the  contractor  will 
retain  ownership,  his  failure  to  provide  these  elements  constitutes  risks  to  the  project 
which  must  be  dealt  with  by  the  government.  The  primary  defense  against  incurring  ex- 
cessive risks  of  this  type  lies  in  the  structuring  of  the  source  evaluation  criteria,  placing 
the  greatest  emphasis  on  technical  expertise  in  both  design  and  production  and  significant 
emphasis  on  their  configuration  management  system.  Unfortunately,  the  vagaries  of  con- 
tracting will  occasionally  lead  to  an  award  to  a contractor  weak  in  one  or  both  areas;  the 
only  defense  the  manager  can  provide  himself  is  a combination  of  in-house  resources, 
effective  management  reports  (required  through  the  CDRL),  and  prayer. 

Another  pitfall  exists.  Large  contractors  sometimes  develop  a design  in  one  divi- 
sion and  produce  the  product  in  another  division  located  10  states  away.  Even  though  the 
contractor  will  retain  design  ownership,  the  design  must  be  transitioned  from  the  one  divi- 
sion to  the  other,  and  the  separate  divisions  take  on  the  character  of  different  companies. 
To  provide  for  a good  transition,  the  government  should  require  that  a preproduction 
model  be  produced,  as  a part  of  the  development  contract,  with  the  same  facilities  (and. 
preferably,  same  personnel)  which  will  be  used  for  the  production  contract. 

As  with  the  system  integration  issue,  when  the  contractor-owncd/govemment-owncd 
design  boundaries  are  established,  they  should  be  clearly  established  and  consistent  with 
the  determinations  made  for  the  other  major  issues  and  with  the  criteria  above. 
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Figure  VI-2.  Sample  system  partitioning  with  major  issues  resolved. 
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Production  items  are  specified  by  product  specifications.  M1L-STD-490  defines 
four  different  types  of  product  specifications  for  use  by  DoD  in  seven  different  formats, 
and  while  industry  does  not  necessarily  conform  to  these  formats,  the  different  types 
are  fairly  universal.  The  most  important  types  are  the  functional  (or  performance)  speci- 
fication and  the  fabrication  (or  detailed  design)  specification;  the  remaining  two  types  are 
the  Noncomplex  Item  Product  Fabrication  Specification  (which,  as  the  name  implies,  is 
an  abbreviated  form  of  fabrication  specification)  and  the  Computer  Program  Product 
Specification.  Both  the  functional  and  fabrication  types  may  be  written  in  two  formats 
depending  upon  the  complexity  of  the  item  specified  (ie,  prime  item  or  critical  item); 
additionally,  the  functional  specification  is  used  in  another  format  as  an  Inventory  Item 
Specification.  Which  specification  type  should  be  used  is  a major  issue  which  must  be 
resolved  in  selecting  a procurement  mode. 


FUNCTIONAL  SPECIFICATION 

The  functional  specification  requires  that  the  finished  hardware  meet  size,  weight, 
external  configuration  and  mounting  provisions,  external  interface  requirements,  and  overall 
performance  of  the  item  within  a specified  environment  (often  referred  to  as  “form,  fit, 
function,”  or  F^,  parameters).  When  the  contractor  has  produced  the  hardware,  the  govern- 
ment is  obligated  to  accept  and  pay  for  it  if  it  meets  the  specified  requirements,  regard- 
less of  the  nature  of  the  internal  design.  This  approach  is  utilized  frequently  for  the  pro- 
curement of  expendable,  nonrepairable  items  and  for  readily  available  items  (such  as 
batteries)  for  which  the  government  is  not  concerned  about  internal  configuration.  When 
used  to  procure  repairable  items,  the  procurement  package  should  include  warranty  provi- 
sions, renewable  maintenance  contract  provisions,  and/or  provisions  for  contractor  services 
to  set  up  the  necessary  government  maintenance  capabilities  to  support  the  equipment 
through  its  intended  service  life. 

The  advantages  of  procurements  controlled  by  functional  specifications  are  as 
follows: 

1.  Detailed  design  responsibility  is  clearly  assigned  to  the  contractor.  If  the  item 
fails  to  meet  specifications,  the  contractor  must  alter  the  design  until  specified 
operation  is  achieved. 

2.  There  is  no  design  data  package  for  the  government  to  procure  or  maintain. 

3.  Requirements  for  technical  capability  within  the  government  are  minimized. 
This  is  the  path  of  least  involvement  on  the  part  of  the  government  in  con- 
tracting, contract  monitoring,  etc. 

4.  Standardization  can  be  achieved  among  multiple  sources  through  two-way 
interchangeability  of  products  which  may  differ  internally.  These  multiple 
sources  may  be  exercised  simultaneously. 

The  disadvantages  include  the  following: 

1.  Each  procurement  contains  a development  effort  unless  the  product  is  off-the- 
shelf-unmodified.  Some  time  and  money  are  involved  each  time  the  item  is 
procured  tor  engineering,  changes,  production  learning  curves,  and  debugging. 
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Since  the  contractor  develops  the  product,  companies  without  a development 
capability  (and  accompanying  development  overhead)  are  excluded.  (Of 
course,  when  an  off-the-shelf  product  is  being  purchased  unmodified  or  slightly 
modified,  the  development  costs  (apportioned  into  each  unit  price)  are  shared 
by  the  entire  market,  of  which  the  government  may  be  only  a small  part.) 

2.  Each  time  a procurement  is  made,  the  contractor  who  has  the  least  apprecia- 
tion for  the  total  significance  of  the  specification  and  the  effort  to  accomplish 
the  task  is  most  likely  to  be  the  low  bidder.  This  means  that  the  source  selec- 
tion criteria  must  be  very  carefully  constructed  to  include  mechanisms  to 
demonstrate  contractor  awareness  of  critical  elements  as  well  as  his  capabilities 
to  produce  the  item.  In  a simultaneous  multisource  procurement,  the  quanti- 
ties to  each  successful  bidder  can  be  made  contingent  (within  boundaries  or 

as  determined  by  a formula)  upon  the  relative  performance  of  each  product 
in  preproduction  testing  for  a limited  set  of  critical  parameters  (such  as  re- 
ceiver sensitivity,  reliability,  and  cost). 

3.  The  costs  of  repair  parts  will  tend  to  become  excessive  when  a contractor 
realizes  that  he  is  in  a somewhat  sole-source  position  with  respect  to  his 
equipment  unless  the  total  maintenance  for  the  service  life  of  the  equipment 
is  provided  for  in  the  procurement  contract  or  in  conjunction  with  the  pro- 
curement contract  while  competition  is  still  being  maintained.  A side  issue 
arises  in  that  procurement  funds  and  operations/maintenance  funds  are 
separate  budget  pots,  requiring  the  program  to  resolve  a potentially  politically 
sensitive  issue  on  the  funding  of  the  procurement  prior  to  its  execution. 

4.  Careful  specification  of  all  external  parameters  is  required  to  ensure  true 
interchangeability.  Each  parameter  must  be  specified  within  tolerances;  it  is 
strongly  recommended  that  output  tolerances  be  tighter  than  the  input  toler- 
ances they  interface  with  and  inversely  for  input  tolerances.  Control  second- 
and  third-order  parameters  (timing  relationships,  phase,  out-band  response, 
spurious  emission,  etc);  overlooked  second-  and  third-order  parameters  may 
result  in  marginal  or  unusable  equipment  that  the  government  is  obligated  to 
accept. 


Even  if  the  functional  specification  is  for  a less  complex  item  than  a major  system,  the  sped 
fication  guidance  provided  for  Type  Cla,  MIL-STD-490,  App  VII,  contains  the  elements 
which  should  be  considered  even  if  another  format  is  used.  There  are  several  points  which 
may  be  clarified  for  functional  specifications.  The  following  items  amplity  type  ( la  para- 
graph guidance; 

1.  Maintainability.  Consider  test  equipment/test  point  provisions  and  any  special 
requirements  such  as  use  of  ATE  or  limits  to  CPE  TE  and  the  interlaces  to  be 
provided  for  test  purposes. 


2.  Design  and  Construction.  Specifically  applicable  paragraphs  of  the  general 
equipment  specifications  (MIL-E-1 6400,  M1L-E-5400  etc)  should  be  cited. 
Metric/English  requirements  should  be  called  out. 

3.  Materials.  Processes,  and  Parts.  Include  the  provisions  to  (a)  prevent  the  un- 
necessary use  of  strategic  and  critical  materials,  (b)  prevent  the  use  of  processes 
which  are  known  to  lead  to  an  unsafe  item  in  field  use,  and  (c)  limit  the  use  ot 
proprietary  or  other  nonstandard  parts  as  appropriate  to  the  maintenance/ 
logistic  support  plan  for  the  item. 
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4.  Interchangeability.  Applicable  items  include  type  and  location  of  connectors, 
connector  pin  allocations,  and  interface  hardware  for  mounting,  for  cooling 
connections,  for  power,  for  insert  keying  configurations,  etc. 

5.  Human  Performance/Human  Engineering.  Drawings  of  specific  control  panel 
configurations,  control  operations,  and  lighting  features  should  be  referenced 
when  appropriate. 

FABRICATION  SPECIFICATION 

The  fabrication  specification  requires  the  hardware  to  be  built  in  accordance  with  a 
detailed  design  data  package.  In  this  manner,  “form”  and  “fit”  are  tightly  controlled  and 
“function”  is  implicitly  controlled  by  the  design  capabilities  of  the  hardware  described  by  the 
data  package.  Only  critical  functional  parameters  are  included  to  ensure  that  the  accepted 
hardware  will  be  functional.  A fabrication  specification  is  always  used  for  government-owned 
designs  (and  vice  versa)  (See  DESIGN  OWNERSHIP,  p VI-5).  The  advantages  of  fabrication 
specification  controlled  procurements  are: 

1 . The  government  maintains  strict  configuration  control;  thus,  all  parts  in  the  field 
are  identical  to  their  counterparts  regardless  of  manufacturer  (unless  specifically 
authorized  changes  are  allowed),  lhus,  repair  parts,  training,  test  equipment, 
technical  manuals,  and  other  logistics  elements  can  be  standardized  and  main- 
tained efficiently  and  cost-effectively  at  the  organizational  or  intermediate 
maintenance  levels. 

2.  The  development  costs  and  associated  long  lead  times  are  borne  only  once  and 
the  government  can  exercise  strict  configuration  management  during  design.  1 he 
design  can  be  fabricated  by  any  contractor  with  the  proper  production  facilities 
and  competence  to  use  them.  This  is  a much  broader  source  base  than  that  lor 
functional  specifications.  Also,  contractors  maintaining  production  facilities 
only  do  not  have  the  high  overhead  rate  associated  with  a development  capa- 
bility (usually  an  acceleration  ot  20-30'?). 

3.  Second-  and  third-order  parameters  which  are  inherent  in  the  specified  design 
will  continue  to  play  an  important  but  unknown,  unappreciated,  and  unspecified 
role  in  the  successful  operation  of  the  hardware. 

4.  Lessons  learned  in  the  production  of  the  item  by  one  contractor  may  be  incor- 
porated in.  the  data  package  to  preclude  duplicate  difficulties  on  succeeding 
contracts,  thus  reducing  risks  with  each  procurement. 

5.  Spurious  reprocurements  and  mobilization  requirements  can  be  met  rapidly  and 
without  significant  risk. 

b.  Good  cost,  quality,  and  production  time  standards  can  be  obtained  and  utilized 
on  future  procurements. 

7.  The  in-house  technical  base  is  generally  increased  and  made  available  for  future 
procurements. 

Most  of  the  disadvantages  of  fabrication  specification  procurements  relate  to  the  fact 
that  the  government  owns  the  detailed  design  and  must  bear  the  responsibilities  ot  ownership 
(see  DESIGN  OWNERSHIP,  above).  Some  other  disadvantages  include. 

1 . The  government  must  pay  for  all  development,  validation,  and  qualification  costs 
incurred  in  assembling  the  design  data  package. 
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2.  The  original  developer  must  be  honest,  accurate,  complete,  and  current  in  his 
generation  of  the  design  data  package.  A lack  of  one  of  these  characteristics 
will  show  up  in  a design  validation  phase,  however,  complete  validations  is  tre- 
quently  omitted  (too  expensive).  Even  when  validation  is  performed,  large  costs 
will  be  incurred  correcting  deficiencies. 

3.  The  state  of  the  technology  base  is  inherent  in  the  data  package.  Highly  mobile 
technologies  can  rapidly  outdate  the  design  and  cause  costly,  hard-to-get  spare 
parts  and/or  costly  redesign  of  the  equipment.  It  is  very  difficult  to  accommo- 
date new  capabilities  without  costly  design  changes. 

4.  Because  functional  parameters  are  largely  not  available,  it  is  very  difficult  to  create 
a standardization  program  which  utilizes  the  item  in  ever-expanding  applications 
and  updates  the  capabilities  to  conform  to  dynamic  operational  needs. 

SELECTION  CRITERIA 

The  choice  between  specification  types  to  support  procurement  is  based  on: 

1.  The  need  for  detailed  design  ownership.  (Detailed  design  ownership  implies 
fabrication  specification,  t 

2.  The  size  of  procurement  and  reprocurement  requirements,  (l  or  items  requiring 
development,  functional  specifications  are  best  applied  to  large  procurements 
where  multisources  can  be  utilized  simultaneously;  lubrication  specifications  are 
valuable  to  support  intermittent  and  spurious  reprocurement  requirements.) 

3.  The  maturity  of  product  design.  (Functional  specifications  are  best  applied  to 
mature  (on  shelf)  designs  which  can  be  used  as  is  or  with  minor  modifications.  I 

In  either  case,  product  specifications  are  intended  for  use  with  fixed-price  contracts.  It  t he 
item  is  repairable,  the  maintenance  of  items  procured  on  functional  specifications  should  be 
included  in  the  contract;  when  this  is  not  possible,  a fabrication  specification  may  be  a belter 
choice. 

Picking  one  specification  type  over  the  other  has  long-term  effects  which  must  be 
dealt  with.  Fabrication  specifications  are  effective  in  establishing  standardization  and  con- 
figuration control  only  at  the  piece  part  level;  this  tact  may  be  inconsistent  with  the  resolution 
of  the  standardization  and  level-of-repair  issues.  Standardization  above  the  piece  part  level  is 
best  supported  by  functional  specifications;  therefore,  functional  specifications  should  be 
established  at  each  level  of  complexity  down  to  the  level  of  standardization  regardless  ot  the 
specification  type  used  for  procurement. 

Occasionally,  procurements  are  observed  in  which  the  government  owns  a detailed 
design  data  package  but  lacks  confidence  in  the  accuracy  and  completeness  of  the  design.  It 
has  been  common  practice  to  furnish  the  drawings  to  the  contractor  "for  information  only ." 
and  require  them  to  retain  interchangeability  with  the  depicted  design  to  a specified  level. 

Ibis  approach  amounts  to  procurement  without  specification,  since  functional  specification 
is  not  established  and  the  government  disclaims  responsibility  for  the  adequacy  ot  the  base- 
line designs.  I he  contractor  assumes  high  risks  which  are  translated  into  virtually  certain 
failure  to  meet  technical,  cost,  or  schedule  goals.  None  ot  the  advantages  ot  either  specifica- 
tion type  are  realized. 

The  application  of  functional  specifications  to  procurements  of  small-lot.  government- 
peculiar  items  requiring  significant  development  is  observed,  also.  Primarily,  the  manager  is 
attempting  to  procure  the  item  without  paying  for  the  design  package.  I his  ploy  works  until 
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a reprocurement  of  even  one  more  item  is  required;  then  the  contractor,  who  is  in  a sole- 
source  position,  must  be  paid  again  for  development,  tooling,  etc.  Usually,  the  procurement 
situation  arises  out  of  an  expensive  and  complex  development  effort  in  which  small  quantities 
are  procured  initially  but  large  quantities  are  eventually  required  but  in  a number  of  small- 
quantity  reprocurements.  In  this  situation,  the  government  should  buy  reprocurement  rights 
with  the  initial  procurement  which  guarantee  that  future  procurements  will  be  met  at  the 
same  unit  cost  (plus  escalation  for  inflation)  as  the  first  units  and  within  a specified  time.  In 
this  way,  the  government  pays  the  contractor  to  maintain  the  design  documentation  and 
production  tooling  until  it  is  needed.  Essentially,  the  government  has  bought  use  of  the  design 
without  obtaining  title  for  custody  to  the  data  package.  This  ploy  is  not  nearly  as  flexible  as 
obtaining  design  ownership  out  of  the  development  phase  and  is  harder  to  implement  on  a 
firm  legal  basis,  but  costs  can  be  significantly  less  in  development.  The  tactic  may  not  be  used 
for  process-sensitive  procurements  because  the  process  yield  will  be  lost  between  buys. 

Functional  specification  implies  contractor-owned  detailed  design,  and  fabrication 
specifi^tion  implies  government-owned  detailed  design.  This  is  fine,  but  who  should  be 
respo^^mle  for  developing  the  specified  parameters?  In  functional  specifications,  the  param- 
eters are  generated  by  the  government  from  the  requirements;  additional  functional 
parameters  may  be  added  to  extend  the  specification  to  a lower  level  of  complexity  by 
adopting  established  standard  interfaces  or  by  procuring  additional  functional  specification 
parameters  from  the  contractor  and  verifying  them  during  qualification  tests.  This  latter 
ploy  is  slightly  more  risky  because  second-  and  third-order  parameters  are  easier  to  overlook. 
Fabrication  design  data  development  is  very  much  more  complicated  and  controversial. 

The  final  subissue  related  to  specification  type  involves  the  level  of  specification; 
ie,  the  level  of  complexity  to  which  the  system  is  specified.  Functional  specifications 
should  be  established  to  the  lowest  level  of  complexity  consistent  with  the  other  major 
issue  resolutions;  fabrication  specifications  should  be  established  consistent  with  the  pro- 
curement considerations  and  design  ownership  boundaries.  The  specifications  should  be 
complete  to  the  design  ownership  boundaries  prior  to  starting  procurement  actions. 


LEVEL  OF  REPAIR  PERFORMED  AT  THE  FIELD  LEVEL 

The  issue  of  level  of  repair  performed  at  the  field  level  should  be  resolved  before 
the  transition  to  production  is  commenced.  The  field  maintenance  level  includes  the  or- 
ganizational maintenance  levels  and  the  field/afloat  intermediate  maintenance  level  The 
level  of  repair  (LOR)  ie.  the  level  of  system  complexity  to  which  the  system  is  repaired 
performed  at  the  organizational  level  should  be  determined  during  concept  formulation  in 
conjunction  with  level  of  maintenance  capability  constraints.  These  constraints  tend  to 
force  the  LOR  at  the  organizational  level  toward  the  system  level  of  complexity;  however, 
practical  constraints  (size,  weight,  item  cost,  and  failure  rate)  tend  to  drive  the  LOR 
toward  the  piece  part  level  The  balance  of  these  forces  tends  to  resolve  the  system  main- 
tenance philosophy  at  approximately  the  unit  (or  WRA)  level  of  complexity.  However, 
the  best  level  at  which  to  discard  failed  items  is  usually  simpler  (such  as  the  module  or 
piece  part  level)  than  the  level  of  organic  repair.  In  order  to  minimize  downtime  and  in- 
ventory quantities,  some  form  of  intermediate  maintenance  activity  is  usually  employed 
to  effect  repair  of  the  unit  at  a SRA  level  and/or  to  establish  a rotatable  equipment  pool. 
A combination  of  noneconomic  analysis  and  an  initial  support  analysis  based  in  the  system 
^ life-cycle  cost  model  is  completed  to  determine  the  level  of  repair  at  the  intermediate 
activities.  After  the  product  design  is  relatively  conq  . ,e.  a level  of  repair/logistics  sup- 
port analysis  performed  as  a portion  of  the  ILS  tasks  is  completed  to  confirm  and  modify 
appropriately  the  initial  determination. 
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Although  tedious,  the  economic  analysis  is  relatively  straightforward  if  a viable  sys- 
tem life-cycle  cost  model  has  been  established:  guidance  is  available  from  M1L-STD-1390, 
MIL-STD-1388,  and  system  effectiveness  personnel  (such  as  those  in  NOSC’s  Product  Assur- 
ance Division).  Noneconomic  analysis  may  or  may  not  play  a significant  role  in  the  LOR 
decision. 

The  noneconomic  analysis  will  consist  of  recognizing  preempting  factors  such  as 
safety,  repair  feasibility,  standardization,  allowable  downtime,  and  the  other  non-cost 
constraining  factors.  If  this  analysis  produces  a definitive  decision  for  LOR,  the  economic 
analysis  is  still  completed  to  formulate  repair/discard  decisions  on  the  basis  of  the  design. 
Usually,  a definitive  decision  will  not  be  reached  until  the  product  design  is  nearly  com- 
plete. Nevertheless,  factors  which  affect  the  noneconomic  analysis  do  impact  upon  the 
build,  buy,  modify  alternatives,  the  system  integration  issue,  and  the  ability  to  achieve 
service  approval.  Of  these  factors,  allowable  downtime  and  standardization  considerations 
tend  to  be  the  confusing  factors.  Standardization  is  itself  a major  issue  (see  STANDARD- 
IZATION, below);  downtime  is  discussed  in  the  following  paragraphs. 

The  downtime  allowable  for  the  system  as  a whole  is  determined  by  its  criticality 
to  the  platform  missions;  the  system  is  classified  as  vital,  semivital,  or  nonvital  with  a 
system  availability  assigned  accordingly.  Downtime  and  mission  reliability  are  the  lactors 
which  make  up  availability;  the  formulation  normally  is  of  the  form 

MTBF 

A MTBF  + MDT 

Not  all  failures  which  can  occur  affect  mission  reliability,  and  the  repair  of  those  failures 
does  not  necessarily  cause  the  system  to  be  down.  Prior  to  the  product  design  phase, 
essential  equipment  items  (those  whose  failure  affects  mission  reliability)  can  be  identified 
at  each  level  of  complexity  within  the  system.  When  the  system  is  partitioned,  many 
(ideally  all)  of  the  essential  items  can  be  lumped  into  a single  cell,  and  at  some  level  of 
complexity  it  is  possible  to  eliminate  the  essential  item  nature  through  voting  techniques, 
redundancy,  etc.  Unfortunately,  it  is  not  always  practical,  even  when  it  is  cost-effective, 
to  partition  in  this  way,  since  other  factors  come  into  play.  However,  a level  of  complexity 
can  be  identified  for  each  portion  of  the  system  for  which  essential  item  effects  are  minimal 
and  for  which  the  allowable  downtime  can  be  reasonably  long  without  affecting  system 
availability.  This  level  allows  effective  intermediate  maintenance  support  with  practical 
turnaround  times  and  is,  therefore,  the  optimum  level  of  repair  from  a downtime  view- 
point. Naturally,  the  final  LOR  determinations  must  take  into  account  many  other  factors. 

When  the  level  of  repair  performed  at  the  field  level  is  identified,  the  information 
is  useful  in  establishing  tradeoffs  between  contractor  support  versus  in-house  support, 
government  configuration  control  requirements,  and  other  (actors  important  to  resolving 
the  other  major  issues.  • j 


STANDARDIZATION 

The  values,  policies,  and  tradeoffs  associated  with  standardization  are  documented 
in  chapter  XIV,  Consideration  of  Standardization.  Level  of  standardization  is  a concept 
which  recognizes  the  benefits  of  standardization  while  acknowledging  that  improper  im- 
plementation can  lead  to  gross  deficiencies  that  negate  the  potential  benefits.  For  a given 
operational  requirement,  it  is  beneficial  to  have  standard  system,  design,  logistics,  etc, 
to  meet  the  requirement.  On  the  other  hand,  logistics  items  (piece  parts,  modules,  etc) 
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must  be  utilized  in  many  systems  to  become  effective  standard  items.  System  designers  ' 
require  flexibility  and  acquisition  managers  require  flexibility,  but  standardization  implies 
inflexibility.  In  order  to  avoid  inflexibility  and  to  be  useful,  standards  must  be: 

• Functionally  specified  (so  interface  data  are  readily  available) 

o Functionally  complete  (can  be  used  as  a building  block) 

• Available  in  minimal  variety  sufficient  to  meet  differing  major  applications 

• Adaptable  (easily  maintained  with  the  state  of  the  art) 

• Well  documented  and  readily  available  to  the  prospective  user 

• Functionally  flexible  (possess  capabilities,  reliability,  etc,  that  make  it  attractive 
to  diverse  applications) 

Standardization  has  strong  implications  for  each  of  the  other  major  issues.  System  par- 
titioning must  be  carefully  constructed  to  use  existing  standards  where  possible  and  to 
make  other  partitioned  portions  of  the  system  attractive  as  new  standards.  The  partitions 
should,  as  much  as  possible,  conform  to  industry  standards  (where  they  exist),  common 
practice,  etc,  to  avoid  “swimming  upstream”  and  to  make  the  item  easier  to  procure.  The 
“standard”  partitions  should  be  consistent  with  design  ownership  and  level-of-repair  deci- 
sions. System  partitioning  which  is  system  peculiar  (unsuitable  for  other  applications) 
should  be  confined  to  as  small  a portion  of  the  system  as  is  practical;  this  puts  limitations 
on  other  partitioning  considerations  such  as  those  for  level  of  repair. 

Functional  specifications  should  be  established  at  each  level  of  complexity  below 
the  system  level  and  above  (but  including)  the  level  of  standardization.  The  level  of 
standardization  will  be  the  worst  case  of  the  design  ownership  boundary  and  the  level  of 
repair.  The  system-peculiar  items  should  be  specified  also,  since  future  efforts  to  stand- 
ardize or  to  change  an  interfacing  standard  may  have  to  identify  critical  interface  para- 
meters. 


MAJOR  ISSUES  SUMMARY 

Each  major  issue  embodies  a family  of  often  sensitive  considerations  which  must 
be  dealt  with  before  a successful  transition  to  production  can  be  completed.  These  issues 
are  closely  related  and  may  not  be  considered  as  isolated  cases  without  risk  of  causing 
dire  perturbations  to  the  others.  Many  of  the  steps  which  must  be  taken  to  resolve  (or 
resulting  from)  these  issues  involve  time  and  money  resources  which  may  appear  to  be 
unnecessary  when  viewed  in  isolation;  however,  failure  to  execute  these  steps  within  the 
coordinated  framework  of  the  major  issues  will  lead  to  greater  overall  program  costs. 


THE  “BUY”  ALTERNATIVE 

The  buy  alternative  describes  the  use  of  existing  equipments  and  presumes  that 
the  equipments  are  readily  available  and  in  use,  and  that  some  use  history  might  be  obtain- 
able. Such  equipments  will  be  identified  by  the  source  search  and  screen  approach  integral 
to  the  TELCAM  technique  (see  chapter  XI,  Screening  Techniques).  This  screening  serves 
to  establish  qualified  equipments  but  does  not  qualify  the  equipment  for  service  use  direct- 
ly; rather,  the  equipments  must  be  integrated  into  the  system  and  the  system  support 


plan  modified  as  necessary  to  accommodate  the  item  (the  modifications  must  remain  within 
the  system  constraints).  For  the  portions  of  the  system  utilizing  this  alternative,  the  major 
issues  will  be  resolved  as  follows: 


Commercial  Equipment 


Military  Equipment 


Government  responsibility  from  the  equipment  level  on  up 
Supplier  As  previously  established 

Functional  Inventory  item 

(functional) 

At  the  equipment  level  No  lower  than  that  pre- 

viously established 

At  the  equipment  level  As  previously  established 

“As  previously  established”  implies  that  this  information  is  available  and  compatible  with 
system  requirements;  incompatibilities  may  often  be  resolved  by  treating  the  military  equip- 
ment as  a commercial  equipment.  The  buy  alternative  flow  is  illustrated  in  figure  VI-3. 

A sophisticated  “buy”  technique  which  is  suitable  for  large,  multiple-buy  procure- 
ments (such  as  avionics)  is  documented  in  AR1NC  Publication  1313-01-1-1447,  Application 
of  the  Commercial  Airline  Acquisition  Methodology  to  Department  of  the  Navy  Electronic 
Equipment  Acquisitions,  15  October  1975,  AD/A015694. 


System  Integration 
Design  Ownership 
Specification  type 

LOR 

Standardization 


THE  “MODIFY”  ALTERNATIVE 

The  modify  alternative  is  usable  when  nonmajor  modifications  can  make  an  existing 
equipment  suitable  for  the  system  application.  The  same  steps  and  procedures  apply  as 
for  the  buy  alternative  except  that  a modification  step  is  added.  The  modification  may 
be  performed  by  the  government  or  by  the  supplier  for  commercial  equipments  depending 
upon  the  support  provisions  in  the  procurement  contracts  and  the  nature  of  the  modifi- 
cations. Modifications  which  can  be  accomplished  externally  to  the  equipment  in  the 
interfacing  hardware  will  normally  be  done  by  the  government;  all  other  modifications 
are  usually  accomplished  by  the  supplier.  The  modification  is  controlled  by  detailed 
design  when  accomplished  by  the  government  and  by  incorporation  into  the  procurement 
specification  when  done  by  the  supplier.  The  two  modification  alternative  (lows  are  illus- 
. trated  in  figure  VI-3. 


THE  "BUILD”  ALTERNATIVE  (DEVELOPMENT) 

The  build  alternative  should  be  pursued  only  when  suitable  buy  and  modify  alter- 
natives are  not  available.  Unfortunately,  the  many  unique  military  requirements  and 
rapidly  evolving  military  technology  force  a development  situation;  even  where  otherwise 
suitable  equipments  exist,  the  required  maintenance  environment  may  be  such  that  the 
equipment  cannot  be  adapted  to  it. 

Development  is  a very  much  more  complex  process  than  the  processes  of  the  buy 
and  modify  alternatives;  therefore,  there  are  more  issues  to  be  resolved,  more  things  that 
can  go  wrong,  more  resources  required,  more  time  needed,  and  more  risks  which  must  be 
assumed  and  managed.  Whereas  each  of  the  major’issues  is  largely  resolved  for  the  other 
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Figure  VI-3.  “Buy"  and  “modify"  How. 
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alternatives,  the  resolution  of  these  issues  is  not  automatic  and  can  be  critical  to  the 
success  of  the  build  alternative;  also,  the  complexity  of  development  includes  some  issues 
peculiar  to  the  process.  These  factors  make  build  the  least  desirable  alternative.  It  is 
necessary  to  separate  requirements  from  desirements,  to  ensure  requirements  validity,  and 
to  research  possible  alternatives  which  can  lead  to  a buy  or  modify  decision  if  the  time, 
expense,  and  risk  of  development  are  to  be  avoided;  a strong  validation  phase  is  very 
important  for  this  reason  (see  chapter  V,  Validation  Phase).  The  purpose  of  the 
transition  to  production  is  to  obtain  a product  for  application  to  service  needs;  expansion 
of  the  technology  base  is  an  exploratory  development  function  which  only  increases  the 
risks  incurred  when  accomplished  during  full-scale  development. 

Before  entering  into  development,  the  system  must  be  partitioned;  this  should 
normally  be  done  in  such  a way  that  portions  subject  to  the  buy  or  modify  alternatives 
can  be  independent  of  the  developed  portions  as  much  as  possible.  Partitioning  along  such 
lines  is  normally  compatible  with  the  resolution  of  the  major  issues;  but  should  conflict 
occur,  partitioning  to  segregate  the  buy/modify  portions  from  the  build  portions  should 
normally  be  given  precedence  unless  the  system  life-cycle  cost  mode!  proves  that  develop- 
ment of  the  portion  in  question  is  a cheaper  alternative  (although  rare,  the  situation  can 
occur).  The  remainder  of  this  section  deals  exclusively  with  development. 

There  are  two  issues  peculiar  to  development  (in  addition  to  the  five  major  issues 

above): 

• The  development  alternative  to  pursue 

• The  transition-to-industry  point 

The  issues  are  interrelated  and  are  primarily  related  to  the  resolution  of  the  system  inte- 
gration and  design  ownership  issues.  The  level  of  repair  should  be  resolved  at  the  most 
cost-effective  level  at  which  it  is  feasible  within  the  mission  constraints.  The  specification 
type  will  be  dictated  by  design  ownership  and  the  selected  procurement  mode,  and  the 
standardization  issue  will  be  determined  by  the  system  partitioning,  design  ownership,  and 
level-of-repair  issues.  The  selection  of  the  development  alternative  should  be  based  pri- 
marily on  the  management  of  perceived  risks  and  on  the  maintenance  of  competition 
(when  the  development  is  accomplished  by  industry);  however,  the  decision  may  be  mod- 
ulated by  the  other  issues.  Transition  to  industry  emjbodies  these  considerations  plus  a 
great  deal  of  controversy  and  political  risk. 

Only  under  the  special  circumstances  outlined  in  chapter  XVI 1,  Development  Alterna- 
tives, can  a development  be  pursued  in  house;  these  same  criteria  may  be  applied  to  the 
production  phase  with  the  result  that  industry  will  almost  always  be  the  source  of  production 
resources.  Onse  the  transition  to  industry  has  taken  place,  the  reverse  transition  should  not 
occur  unless  new  production  resources  must  be  recruited.  Usually,  the  reverse  transition  is 
only  a temporary  one  to  effect  transition  from  a sole  source  to  a multisource  situation  (ret 
NAF1  TR-1 901 );  a permanent  reverse  transition  normally  occurs  only  when  the  government 
is  dependent  on  an  item  in  which  competent  industry  sources  are  no  longer  interested. 

The  reverse  transition  serves  to  develop  a data  package  in  which  the  government  has 
high  confidence;  it  can  be  avoided  if  proper  care  is  taken  in  procuring  adequate  documenta- 
tion initially. 

The  transition  may  occur  at  the  beginning  of  the  acquisition  cycle  as  in  the  case  in 
which  a system  is  wholly  dependent  on  a technology  which  has  been  developed  within  industry, 
or  it  may  occur  following  pilot  production  (and  coinciding  with  the  release  to  production 
decision).  Or  both  transitions  may  take  place  with  a reverse  transition  occurring  alter  engi- 
neering development.  Other  transition  points  are  possible  (see  fig  VI-41. 


VI-18 


JUN  77  J H TOWNSEND 

UNCLASSIFIED  NOSC/TD-108  NL 

ADA044790 


SEE  CHAPTER  X 

1 


Figure  VI-4.  “Build"  (develop)  (low. 


If  (he  design  ownership  is  going  to  reside  with  industry,  the  transition  to  industry 
point  should  depend  on  the  need  for  competition  in  the  procurement  mode  and  the 
maturity  of  the  design.  If  the  design  constitutes  a major  modification  of  commercial 
products  and  does  not  depend  on  new  technology,  competition  is  relatively  inexpensive 
to  achieve.  However,  if  the  design  depends  heavily  on  new  technology  or  on  technologies 
not  commonly  used  in  established  large-market  products^competition  will  be  limited  by 
program  resources  because  a large  amount  of  high-risk  development  will  be  duplicated  by 
as  many  sources  as  are  needed  to  support  competition  (at  leas  (Two  more  if  the  techni- 

cal risk  is  considered  high).  Competition  is  important  to  reduce  co'?b^chedule  risks  and 
to  maintain  control  of  the  procurement.  The  program  must  have  the  res&»«ces  and  justi- 
fication based  on  usage  to  support  these  conditions;  otherwise,  the  design  ownership  should 
reside  with  the  government. 

In-house  resources  must  normally  be  used  to  develop  adequate  operational  and 
technical  requirements  for  the  specification  used  to  document  the  transition  to  industry. 

Most  frequently,  the  conditions  which  force, the  program  into  development  will 
also  favor  the  government-owned  detailed  design  decision  which  is  supported  by  a fabri- 
cation-type product  specification.  The  transition  to  industry  point  will  depend  initially 
on  whether  in-house  resources  or  industry  sources  are  to  be  utilized  for  development  (see 
the  discussion  below).  In  either  case,  a competitive  development  may  be  indicated  to 
choose  between  different  technical  approaches  which  are  feasible.  If  industry  has  developed 
a technology  on  which  the  equipment  is  primarily  dependent,  the  same  industry  sources  will 
probably  be  used  through  engineering  development.  If  in-house  resources  have  been  utilized 
through  the  proof  of  feasibility  (advanced  development),  a probable  transition  point  is  at  the 
entry  into  engineering  development. 

The  primary  purposes  of  engineering  development  are  product  design  and  produc- 
tion engineering;  ie,  those  tasks  which  make  an  item  readily  reproducible.  The  functional 
design  of  the  equipment  should  be  complete  within  a high  confidence  level,  although  it  is 
necessary  for  all  portions  of  the  design  to  have  been  reduced  to  hardware.  Therefore, 
some  low-risk  design  tasks  may  be  incorporated  into  engineering  development  in  addition 
to  the  inherent  design  tasks;  furthermore,  tasks  involving  any  computer  software  associated 
with  the  equipment  must  be  integrated  into  the  El)  effort.  Also,  the  EDM  is  the  first 
level  at  which  the  design  is  sufficiently  mature  that  the  1LS  tasks  can  be  completed.  The 
coordination  of  these  efforts  requires  effective  management  control  by  the  government  of 
the  entire  task  package;  the  system  integration  function  is  a primary  means  of  coordina- 
tion and  control. 

Many  instances  of  serious  problems  in  engineering  development  can  be  cited. 

While  failure  to  meet  technical,  cost,  and  schedule  targets  is  the  result,  the  root  problem 
can  usually  be  traced  to  one  or  more  of  the  following  actions: 

1.  Proceeding  into  engineering  development  without  a complete  functional 
design  of  a high  confidence  level;  this  constitutes  an  incomplete  validation 
of  the  technical  approach. 

2.  Failure  to  implement  an  effective  configuration  control  program. 

3.  An  attempt  by  the  government  manager  to  contract  system  engineering/ 
integration  responsibilities  (at  the  system  level,  this  is  virtually  impossible  for 
a contractor  to  perform  and  results  in  not  meeting  the  responsibilities  at  all). 
Commonly,  many  GET  items  will  be  included,  each  with  its  own  schedule  risk, 
thus  greatly  compounding  the  program  risk  by  instituting  a large  number  o!  un- 
necessary interdependent  risk  elements.  A failure  in  one  aftects  the  entire  system. 
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4.  failure  of  the  government  to  procure  and  validate  the  required  design  data. 

5.  Failure  to  properly  partition  and  specify  the  system  to  the  required  level  of 
complexity  within  elements  contracted  to  industry  (specification  is  required 
to  at  least  the  intended  level  of  repair). 

6.  Failure  to  identify  and  support  the  in-house  technical  talent  required  to  moni- 
tor and  control  contractual  development  efforts. 

7.  Selecting  industry  where  in-house  resources  are  clearly  appropriate.  Virtually 
all  the  actions  listed  above  are  made  (1)  “to  reduce  program  costs"  or  (2) 

to  conform  to  a policy  (often  unstated)  requiring  “early  contractor  involvement. 

The  effort  to  reduce  costs  usually  arises  from  a failure  to  estimate  and  budget 
adequately  at  the  inception  of  the  program.  Later,  as  the  program  approaches  the  FI) 
phase,  accurate  estimates  are  viewed  as  “cost  overruns”  by  budgeteers.  Once  a grossly 
inadequate  estimate  is  made  and  accepted,  a very  large  risk  of  program  cancellation  exists, 
and  it  grows  with  time;  the  situation  can  often  be  resolved  only  by  very  strong  justifica- 
tions including  a detailed  comparison  of  both  estimates  and  by  presenting  a program 
schedule  which  will  not  impact  the  budget  figures  in  the  next  2 fiscal  years.  On  the  other 
hand,  gross  overestimates  at  program  inception  will  normally  result  in  not  establishing  a pro- 
gram at  all  because  relatively  less  justification  is  available  so  early.  Unless  detailed,  high- 
confidence  estimates  can  be  made,  it  is  better  to  make  no  estimate  at  all  or  to  make  highly 
qualified  estimates  in  conjunction  with  a program  plan  which  allows  for  budget  revisions. 
Attempts  to  cut  costs  by  excising  such  “luxuries"  as  in-house  technical  support  only  eliminate 
the  tools  necessary  to  manage  and  manipulate  the  risks  present  in  any  program.  Without 
these  tools,  risks  would  become  interdependent,  and  a failure  in  one  risk  element  would 
create  a chain  reaction  of  failures. 

The  in-house  versus  contractor  design  development  controversy  has  its  basis  in 
the  national  policy  that  the  government  does  not  compete  with  industry.  Therefore,  even 
when  in-house  resources  are  capable  of  handling  the  task  at  costs  below  those  of  industry, 
industry  must  be  selected.  However,  in-house  resources  are  justified  when  the  ability  to 
meet  the  requirements  is  jeopardized  by  any  industrial  alternative.  It  is  important  to 
realize  that  advances  in  the  state  of  the  art  are  not  sought  during  the  product  development 
phase;  these  are  accomplished  through  exploratory  development.  The  following  conditions 
comprise  valid  justifications  for  in-house  development: 

1.  No  qualified  industrial  interest  expressed  in  the  development. 

2.  National  security  precludes  contractual  development. 

3.  The  technical  requirements  are  too  closely  related  to  user  interactions. 

4.  The  technical  requirements  interact  closely  with  service  doctrine. 

5.  The  government  cannot  develop  specifications  adequate  for  contract  and 
cannot  otherwise  obtain  qualified  contract  services  within  the  required  time 
frame. 

6.  The  system  characteristics  and  development  circumstances  involve  a special 
case  in  which  in-house  expertise  far  exceeds  industrial  expertise.  Where 
national  security  or  a lack  of  qualified  industrial  interest  is  involved,  the  gov- 
ernment has  little  choice  but  to  develop  in  house.  The  remaining  situations, 
although  significant,  do  not  ol  themselves  preclude  the  option  for  contractual 
development,  but  it  is  important  to  recognize  that  greater  risk  and  reduced 
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cost-effectiveness  would  probably  result  from  exercising  this  option.  Where 
the  technical  requirements  closely  interact  with  the  user  and/or  service  doc- 
trine, or  where  they  cannot  be  adequately  specified,  a large  number  of  changes 
are  likely  to  occur  during  development.  In-house  development  is  significantly 
more  responsive  to  change  with  much  less  cost/time  impact  than  is  contract 
development  in  fixed-price  contractual  arrangements  (cost-plus  contracts 
cannot  be  used  where  stringent  contractual  controls  must  be  exercised  or 
the  development  cost/time  is  severely  constrained). 

Whether  the  design  is  developed  by  industry  or  in  house,  it  must  still  be  validated: 
however,  in-house  designs  are  easier  and  less  risky  to  validate  because  they  are  inherently 
nonproprietary  and  more  likely  to  be  subject  to  configuration  control  meeting  the  govern- 
ment’s requirements. 

Policies  requiring  “early  contractor  involvement”  usually  originate  within  the  sys- 
tems commands  either  in  written  form  or  in  an  unstated  form  reflecting  a normal  routine 
of  business.  There  are  no  stated  policies,  directives,  or  instructions  above  the  systems 
command  level  supporting  this  position.  Indeed,  the  Navy  RDT&F.  Management  Guide 
(para  0624)  states,  “A  series  of  actions  to  contract  out  important  activities,  each  wholly 
justified  when  considered  on  its  own  merits,  may,  when  taken  together,  erode  the  Govern- 
ment’s ability  to  manage  its  research  and  development  programs.”  The  decision  whether 
to  involve  industry  should  be  made  on  the  basis  of  the  need  for  services  which  are  best 
provided  by  industry,  the  policy  that  the  government  does  not  compete  with  industry, 
and  the  needs  of  the  government  to  maintain  control  of  its  development  efforts  and  to 
execute  its  system  integration/design  ownership  responsibilities  (these  needs  do  not  consti- 
tute competition  with  industry  when  they  dictate  in-house  development).  Where  indicated, 
in-house  production  engineering  capabilities  reside  at  Naval  Avionics  Facility,  Indianapolis 
(NAFI)  (ref  NAFI  TR-1873). 

Another  possible  transition  point  is  following  the  ED  phase  and  entering  prepro- 
duction. The  purpose  of  the  preproduction  phase  is  to  validate  the  design  data  package 
through  a limited  or  pilot  production.  Changes  indicated  by  the  results  of  engineering 
tests  on  the  EDM  may  be  incorporated  as  well  as  changes  which  result  from  producibility 
improvements.  The  validation  or  proofing  function  is  sometimes  overlooked  because  the 
EDM  contractor  is  allowed  to  do  the  preproduction;  furthermore,  the  preproduction  phase  is 
sometimes  used  to  correct  gross  deficiencies  in  the  EDM  which  totally  obscure  the  objectives 
of  preproduction.  In  other  words,  preproduction  is  sometimes  used  to  correct  this  sins 
committed  in  engineering  development  rather  than  to  validate  a finished  design  package, 
which  is  its  intended  purpose.  The  nature  of  the  textbook  preproduction  phase  is  such  that, 
if  a design  data  package  is  ever  going  to  be  used  for  competitive  procurement  or  by  a facility 
other  than  that  producing  the  design,  it  must  be  executed  through  unbiased  analysis  and 
stringent  change  control  procedures.  Since  the  government  is  assuming  the  responsibility  for 
the  accuracy  of  the  design  package,  it  follows  that  these  functions  should  be  performed  by 
qualified  government  personnel.  This  may  vary  from  government  production  personnel 
acting  as  plant  representatives  for  the  program  to  execution  of  the  entire  preproduction  phase 
in  house.  The  latter  approach  is  certainly  most  appropriate  for  large-quantity  or  high-dollar- 
volume  (greater  than  S20M  in  development)  programs,  and  it  is  the  lowest-risk  approach  to 
the  preproduction  problem.  There  is  also  a preproduction  phase  in  functionally  controlled 
procurements  which  require  development,  but  in  this  situation  the  government  is  only 
ensuring  that  the  production  contractor  can  indeed  produce  his  design  (therefore,  it  is 
important  to  require  that  the  same  facilities  be  utilized  for  preproduction  as  for  production). 


The  optimum  transition  to  industry  tor  in-house  developed  designs  and  for  designs 
validated  in  house  is  following  the  preproduction  phase.  A proofed  data  package  is  available, 
and  manufacturing  specialty  houses  (ie,  no  development  capability  with  significantly  lower 
overhead  costs)  can  be  solicited.  The  in-house  technical  team  is  in  a position  to  provide 
strong  support  to  aid  and  monitor  the  contractor,  and  the  government  has  configuration  con- 
trol. The  system  integration  is  complete  and  within  the  control  of  the  government.  The 
procurement  is  on  a fixed-price  basis.  The  technical,  cost,  and  schedule  risks  are  all  minimal. 
Since  release  to  production  cannot  take  place  until  Approval  for  Service  Use  (see  chapter  VII) 
is  issued,  there  is  ample  time  to  incorporate  changes  dictated  by  the  NTE/OPEVAL,  to 
negotiate  the  procurement  package,  and  to  conduct  first-article  tests.  If  the  transition  is 
delayed  until  after  NTE/OPEVAL,  a long  administrative  lead  time  must  be  tolerated  before 
the  initial  equipments  are  available  for  tleet  introduction:  this  transition  point  is  only 
recommended  for  small  quantities  being  procured  from  one  source  at  a time. 

However  the  transition  to  industry  is  performed,  the  risks  are  minimized  at  each  step 
if  the  objectives  of  the  succeeding  steps  are  made  to  influence  the  technical  effort.  This 
means  strong  system  engineering  control  within  the  government  and  contractural  incentives 
on  industry  involvement. 


PROCUREMENT  ALTERNATIVES 

The  offices  and  agencies  of  the  government  have  evolved  a seemingly  limitless  variety 
of  procurement  practices.  In  general,  what  has  always  been  practiced  by  a group  tends  to  be 
what  is  practiced  on  any  procurement  by  that  group  regardless  of  the  circumstances.  When 
the  group  is  dealing  with  a limited  scope  of  products  and  with  the  same  set  of  circumstances, 
this  blind  approach  has  generally  proved  successful;  however,  alterations  in  the  procurement 
circumstances  usually  lead  to  cost  overruns  and  inferior  products  unless  the  procurement 
practices  are  altered  appropriately.  This  section  deals  with  the  alternatives  which  are  available 
to  tailor  the  procurement  method  to  the  circumstances. 

The  primary  procurement  variable  is  that  which  is  to  be  procured.  Various  potential 
suppliers  are  organized  in  a number  of  different  ways  in  order  to  promote  the  greatest 
efficiency  and  least  risk  in  creating  their  primary  products:  the  procurement  method  should 
favor  those  suppliers  which  are  in  the  best  position  to  supply  the  product  of  interest.  The 
product  consists  of  the  end  item  (component,  equipment,  system,  etc)  and  of  a level  of  service. 
The  services  which  may  make  up  part  of  a product  are: 

Research 
Development 
Production 
Support  services 

The  level  of  service  required  for  a product  is  the  most  significant  factor  in  the  determination 
of  a procurement  method.  Each  service  capability  requires  special  resources;  the  maintenance 
of  those  resources  is  a burden  on  the  company’s  cost  of  doing  business.  All  suppliers  are 
primarily  selling  production  services  and  maintain  other  services  in  support  of  those  efforts. 

In  general,  the  greatest  investment  return  is  in  providing  production  services.  On  the  other 
hand,  research  generally  provides  no  direct  return  on  the  investment:  however,  very  substantial 
returns  are  generally  realized  when  a research  product  becomes  successful  in  production. 
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The  range  of  services  provided  differs  substantially  among  the  various  sectors  of 
suppliers.  In  the  “consumer”  sector  of  many  products,  companies  are  geared  tor  production 
and  offer  extensive  support  services  through  warranties,  field  representatives,  and  local  repair 
activities:  development  capabilities  are  generally  minimized  to  those  necessary  to  maintain  a 
competitive  product  line.  The  “industrial”  sector  companies  are  similar  to  their  consumer 
sector  counterparts  except  that  a research  capability  and  an  expanded  development  capa- 
bility are  maintained  to  support  high-technology  product  lines.  In  both  consumer  and 
industrial  sector,  high-volume  production  is  the  goal  and  the  key  to  product  prol itability. 

The  defense  sector  is  ditterent. 

The  defense  sector  is  characterized  by  intensive  research  and  development  and  by  low 
production  volumes,  since  the  products  are  often  military-peculiar  and  required  in  small 
quantities.  The  companies  participating  in  the  defense  sector  have  well  established  R&l) 
capabilities,  limited  production  capabilities  which  are  set  up  specitically  lor  low-volume 
production,  and  virtually  nonexistent  support  services,  l.ven  companies  which  participate 
heavily  in  both  industrial  and  defense  sectors  are  usually  organized  to  segregate  the  industrial 
and  defense  capabilities,  and  each  part  conforms  to  the  characteristics  ol  its  sector. 

In  recent  years,  high-technology  industrial/consumer  markets  have  seen  the  introduc- 
tion of  many  products  which  are  readily  adaptable  to  military  applications,  and  a great  deal 
of  pressure  has  been  applied  to  Dol)  to  use  both  commercial  end  items  and  services,  especially 
warranty  services.  T he  commercial  sector  is  not  prepared  for  the  extensive  documentation 
and  quality  assurance  provisions  nor  for  military  standard  parts  requirements  which  are 
common  to  many  Dol)  procurements,  l ikewise,  the  detense  sector  is  not.  in  general,  pre- 
pared to  deal  with  long-term  warranty  clauses  nor  tield  service  provisions,  and  large  produc- 
tion runs  will  often  have  higher  unit  costs  than  non-DoD  sector  production  runs. 

1'he  alternatives  which  should  be  considered  to  make  the  most  ellicient  use  ot  private 
sector  talents  include  development  alternatives  which  lead  to  an  open  competitive  production 
(as  opposed  to  closed  competition  which  is  limited  to  the  multiple  developers);  closed-end 
contracts  for  each  program  phase  (the  product  ot  each  contract  is  sulticienl  tor  a ditterent 
contractor  to  perform  the  next  phase):  split-tasking  (each  task  is  pertormed  by  a supplier 
experienced  in  the  required  service):  and  the  use  of  in-house  resources  to  plug  gaps,  as  well  as 
the  traditional  single  supplier  methods.  Neither  closed-end  contracting  nor  split-tasking  is 
always  efficient  because  the  data  requirements  to  effect  handovers  between  contractors  may 
be  excessive  and  impossible  to  obtain  in  the  detail  needed:  each  case  must  be  reviewed  on  its 
own  merits.  Split-tasking  will  tend  to  be  very  ellicient  il  a system  can  be  partitioned  into 
equipments  requiring  development  and  units  which  can  be  purchased.  In  any  case,  the 
procurement  solicitations  should  require  suppliers  to  show  their  management  approach  and 
their  resources  to  apply  toward  any  services  which  would  not  be  in  their  normal  market 
requirements;  this  demands  some  knowledge  of  the  various  probable  responders  to  the 
solicitation. 


TOTAL  CONTRACT  COST 


w 


I'lie  type  ot'  contract  to  he  employed  is  another  procurement  variable.  A Fixed  price 
type  contract  is  most  desirable  as  the  risk  elements  are  most  Firmly  established  and  can  be 
minimized.  However.  Fixed  price  contracts  cannot  be  used  outside  well  established  limits 
(see  chapter  XVI.  Types  oF Contracts).  In  general,  situations  allowing  Fixed  price  contracting 
will  Favor  commercial  supplier-type  services. 

The  selection  of  the  procurement  scheme  to  be  used  must  be  made  in  consideration 
of  the  development  alternative  being  pursued  (see  chapter  XVII.  Development  Alternatives). 

When  the  procurement  quantity  is  low.  the  development  considerations  will  predominate, 
but  high  procurement  quantities  demand  attention  to  the  potential  costs  oF  each  develop- 
ment procurement  combination.  Getting  industry  involvement  early  in  the  program  may  he 
important  to  the  ultimate  cost  of  the  product.  Obviously,  pursuing  a development  alternative 
which  transitions  the  technical  tasks  to  industry  early  in  the  program  gets  industry  involved: 
however,  only  the  limited  and  parochial  views  of  one  company  or  a Few  (For  parallel  develop- 
ments) companies  may  be  obtained  rather  than  an  industry-wide  view.  Another  method  uses 
multiple-step  procurements  which  obtain  industry  views  through  the  Formal  procedures  ot' 
solicitation,  proposal,  and  evaluation-negotiation:  multiple  step  procurements  can  only  obtain 
a limited  amount  of  precontract  industry  involvement  because  each  company  has  limited 
time  to  review,  react,  and  respond  and  because  the  legal  restrictions  can  hamper  an  open 
technical  exchange.  Under  some  circumstances,  the  various  industry  associations  can  provide 
limited  information  which  can  guide  a development-procurement  approach:  on  large  projects, 
this  method  can  be  effective  when  it  is  supplemented  by  “independent”  studies  of  industrial 
capabilities,  processes,  and  limitations.  Another  approach  which  gets  industry  involved  is  an 
adaptation  of  the  commercial  airlines  acquisition  methodology.  1 his  method  can  only  be 
applied  under  certain  well  defined  circumstances  including  large  in-service  quantities,  multiple 
buys,  and  mature  technologies,  but  its  advantages  include  very  low  development  dollar 
involvement  and  competitive  incentives  maintained  well  into  the  support  phase.  ARING 
Research  Publication  131  .^-0 1-1-1 4-47.  “Application  of  the  Commercial  Airlines  Acquisition 
Methodology  to  Department  of  the  Navy  electronic  equipment  Acquisitions,"  dated 
15  October  ll>75  (AD/A  015  bl)4).  details  the  steps,  advantages,  and  restrictions  of  this 
method. 

When  considering  the  procurement  alternatives,  there  are  three  points  to  remember: 

1 . Be  a good  customer 

2.  Remember  the  ultimate  user 

3.  Maintain  competition  as  long  as  possible 

Particularly  when  working  with  commercial  suppliers,  it  is  important  to  consider  their  normal 
modes  of  operation,  and  to  conform,  as  much  as  possible,  to  the  characteristics  of  their  usual 
market.  Minimize  unusual  requirements  such  as  abnormally  tight  tolerances,  heavy  data  and 
documentation  requirements,  and  extensive  quality  assurance  provisions.  It  is  desirable 
always  to  work  with  the  suppliers  and  to  avoid  adversary  circumstances.  In  working  with 
industry  and  getting  industry  involvement,  do  not  Forget  the  requirements  of  the  ultimate 

user.  It  may  he  necessary  to  trade  off  some  of  the  desirable  characteristics  ot'  the  product  if  J 

they  cannot  be  incorporated  cost-effectively  but  do  not  compromise  anv  of  the  required 
characteristics.  IF  conferences  are  held  to  obtain  industry  involvement,  invite  user  participa- 
tion. also.  ! very  supplier  is  in  business  For  gain  and  is  entitled  to  a Fair  and  reasonable  profit . 
however,  the  government  as  the  buyer  is  entitled  to  a product  with  value.  Competition  is  the 
most  effective  regulator  of  suppliers  in  assuring  a continued  product  with  value.  Whenever 
possible,  maintain  competition  through  multiple  procurements  from  multiple  suppliers  based 
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on  past  experience  of  supplier  performance:  the  details  of  future  procurement  based  on  past 
performance  can  be  established  contractually.  1'his  method  can  obviate  the  need  for  restric- 
tive specifications  used  to  weed  out  potentially  weak  suppliers.  It  is  important,  also,  to  avoid 
requirements  which  are  proprietary  to  one  company  or  which  greatly  advantage  one  company 
over  all  others;  this  is  a particularly  important  point  to  monitor  when  industry  involvement  is 
solicited  because  one  company’s  “good  idea”  can  represent  a proprietary  advantage.  When  it 
becomes  necessary  to  end  competition,  all  future  costs  should  be  fixed  contractually  with 
incentives  and  penalties  established  as  appropriate.  This  is  not  possible  in  total  for  most 
situations,  but  it  is  a goal  to  be  approached.  L.C'C'  and  DTC  procurements  are  examples  of 
contractually  fixing  future  costs. 

The  procurement  approach  is  an  effective  tool  for  reducing  costs  and  promoting 
product  value  when  it  is  selected  to  bridge  the  conditions  of  the  product  development  with 
the  circumstances  of  the  potential  suppliers.  It  should  be  selected  with  attention  to  the  user 
requirements  as  well  as  quantities,  costs,  technical  risk,  and  potential  supplier  capabilities. 
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VI 1 APPROVAL  FOR 
SERVICE  USE (ASU) 


VII.  APPROVAL  FOR  SLRV1CE  USE  (ASU) 


Approval  for  Service  Use  (ASU)  is  granted  only  after  certain  set  conditions 
have  been  met:  ASU  must  be  granted  before  equipments  can  be  introduced 
into  the  fleet.  Two  instructions  are  mandatory  NAVMAT  Instruction 
4720.1  and  OPNAV  Instruction  4720.4  (series).  This  chapter  summarizes 
these  instructions.  Because  the  conditions  for  ASU  are  mandatory,  plans 
to  achieve  them  must  be  integrated  into  project  planning  early  to  ensuie 
timely  approval. 


Approval  for  Service  Use  must  be  granted  before  an  equipment  or  system  can  be 
committed  to  major  production.  ASU  cannot  be  granted  until  the  following  prerequi- 
sites have  been  completed. 

1.  At  least  the  initial  operational  test  and  evaluation  is  complete  and  meth- 
ods for  correction  have  been  confirmed.  A Test  and  Evaluation  Master 
Plan  is  required. 

2.  At  least  the  minimum  performance  requirements  (reliability  and  maintain- 
ability) of  the  approved  developments  proposal  have  been  achieved  and  a 
planned  maintenance  system  has  been  developed. 

3.  Integrated  logistic  support  planning  has  progressed  to  the  point  that  there 
is  asitu ranee  that  all  elements  of  logi^j'  support  will  be  available  in  ap- 
proved form  URpn  delivery  of  the  first  production  item. 

4.  Technical  documentation  necessary  for  support  of  the  system  or  equip- 
ment has  been  identified  and  technical  manuals  have  been  assured  with 
first  deployment  of  production  item. 

5.  Personnel  requirements  are  assured  for  fleet  operation  and  maintenance. 

6.  The  configuration  management  product  base  line  has  been  established  for 
identification  and  future  configuration  changes. 

7.  A system  safety  program  (MIL-STD-882)  has  been  established.  When  ex- 
plosives are  utilized,  a safety  review  and  recommendations  by  the  System 
Explosive  Safety  Review  Board  are  required. 

When  the  above  prerequisites  have  been  met.  the  Systems  Commander  or  CNM- 
designated  Project  Manager  will  prepare  the  ASLI  ; .'commendations.  These  recommen- 
dations are  submitted  to  the  Deputy  Chief  of  Naval  Material  (Operations  and  Logistics) 
(I)CNM  (O&L)). 

The  final  approving  authority  depends  upon  the  magnitude  of  the  project.  Ma- 
jor projects  (RDT&l  greater  than  S50M.  production  greater  than  S200M)  are  approved 
for  ASU  by  the  ('NO  with  the  advice  of  the  CNO  Executive  Board  (CEB).  Less  than 
major  projects  (RDT&E  and  production  less  than  major)  are  approved  for  ASU  by  the 
OPNAV  program  sponsor  with  recommendations  of  the  OPNAV  Service  Review  Board. 
Other  projects  are  approved  for  ASLI  by  DCNM  (O&L)  or  authority  may  be  delegated  to 
a Systems  Commander  or  CNM-designated  Project  Manager. 


Once  an  equipment  or  system  is  ASU,  reupproval  will  not  be  required  unless 

1 . The  equipment  or  system  proves  deficient, 

2.  The  equipment  or  system  is  to  be  used  in  a different  operational  environ- 
ment. or 

3 The  equipment  or  system  undergoes  a significant  alteration.  SYSCOM 
Commanders  and  PMs  notify  DCNM  (O&L)  and  the  system/equipment 
program  support  1CP  if  an  ASU  file  number  is  to  be  canceled  because 
the  equipment  or  system 

I Has  been  modified  or  altered  to  the  extent  that  a new  ASU  has 
been  obtained, 

2.  Is  no  longer  in  use,  or 

3.  ASU  has  been  withdrawn. 

IX’NM  (O&L)  is  responsible  for  the  maintenance  of  records  of  ASU 
actions. 


DEVIATIONS 

In  cases  of  extreme  urgency  or  military  necessity  (such  as  QRC  or  RDC).  pro- 
duction approval  may  be  obtained  in  advance  of  ASU.  SECNAV  (ASN  (l&L))  must 
approve  the  waiver  of  the  ASU,  and.  in  case  of  major  programs,  final  approval  must 
be  obtained  from  the  SECDEF.  All  such  requests  shall  include 

1.  Quantities  to  be  produced  or  procured  and  justification  for  limited  pro- 
duction in  advance  of  ASU, 

2.  Minimum  quantity  needed  to  accomplish  evaluations  on  which  to  base  ASU. 

3.  Analysis  of  all  feasible  alternatives  to  waiver. 

4.  Cost,  schedule,  and  performance  impact  of  each  alternative  on  the  program, 

5.  Results  of  T&E  conducted, 

6.  Proposed  revised  test  plan,  including  plan  for  obtaining  ASU. 

7.  ILS  plans,  and 

8.  Statements  of  risks,  including  alternative  courses  of  action. 

PROVISIONAL  APPROVAL  FOR  SERVICE  USE  (PASU) 

When  sufficient  operational  testing  to  support  a final  determination  of  ASU  can- 
not practicably  be  accomplished  prior  to  making  initial  production  commitments,  a 
PASU  can  be  granted  for  initiating  orderly  first-lot  limited-production  runs. 
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The  procedure  for  obtaining  a PASU  is  the  same  as  the  procedure  for  obtaining 
an  ASU  except  that  the  requirements  on  the  prerequisites  are  less  stringent.  A PASU 
requires  the  following  prerequisites: 


1.  Completion  of  Test  and  Evaluation  Master  Plan  with  corrections  on  defi- 
ciencies to  be  considered  and 

2.  Production  specification  requirements  and  procedures  for  confirming  that 
the  reliability  and  maintainability  will  be  achieved. 
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Introduction  of  equipments  into  the  fleet  is  entirely  determined  by  the 
prior  phases.  The  planning  for  a smooth  phase-in  of  the  new  equipments 
and  phase-out  of  obsoleted  equipments  must  be  accomplished  at  least  2 
years  prior  to  the  actual  task  commencement.  Therefore,  the  tasks  des- 
cribed in  this  chapter  must  be  planned  during  the  validation  phase  or 
early  in  full-scale  development. 

After  the  new  equipments  are  phased  in.  their  performance  should  be 
monitored  to  ensure  that  no  unforeseen  problems  have  been  introduced 
and  to  assure  satisfaction  of  the  operational  requirements.  Deviations 
from  expected  performance  may  require  correction  — at  least  in  future 
acquisitions,  if  not  immediately.  The  knowledge  gained  through  the  ac- 
quisition cycle  should  be  retained  and  applied  to  future  acquisitions,  not 
cast  away;  often  the  greatest  savings  are  those  realized  on  future  projects 
which  can  benefit  from  the  experience  gained  by  project  personnel. 


SUPPORT  REQUIREMENTS 

System  support  elements  include  everything  needed  to  operate  and  maintain  the 
system  over  its  life.  Some  of  these  elements  are: 

Operator  and  maintenance  personnel 

Repair  spare  parts 

Test  equipment  and  tools 

Technical  manuals  (both  operation  and  maintenance) 

Facilities  (depot,  IMA,  overhaul,  calibration,  etc) 

Transportation  and  handling  equipment 

Training  courses  and  materials 

Technical  data  for  provisioning  and  reprocurement 

Technical  data  for  support  management  effectiveness  systems 

Installation  support 

These  elements  may  be  required  in  varying  degrees  over  the  life  of  the  system, 
which  for  support  purposes  is  phased  as  follows: 

Acquisition 

Phase-in 

Operations 

Phase-out 

The  various  support  requirements  for  the  acquisition  phase  vary  widely  and  usu- 
ally differ  markedly  from  those  for  the  other  phases.  The  planning  for  each  portion  of 
the  acquisition  phase  should  be  started  with  the  first  program  planning  effort  and  com- 
pleted prior  to  the  initiation  of  that  portion.  Since  it  is  so  integral  to  program  planning, 
no  further  discussion  is  offered  here. 
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The  phase-in  support  period  starts  with  the  IOC  date  and  ends  whenever  the  lull 
operations  support  is  initiated.  Phase-in  support  is  normally  proeured  at  the  same  time  as 
the  equipment  and  most  often  will  eonsist  of  contractor-supplied  training  and  initial  spare 
parts.  Warranty  serviee  and  maintenance  contracts  also  fall  into  this  support  category. 

The  planning  for  phase-in  type  support  must  be  accomplished  during  the  preproduction 
phase  for  developed  equipments  and  prior  to  the  procurement  solicitation  for  on-shelf  or 
m od.i  tied  eq  u i pm e n t s . 

Operations  phase  support  is  the  normal  service-provided  support  which  lasts  most 
of  the  service  life  of  the  equipment.  The  operations  phase  support  requirements  are  ini- 
tially predicted  by  the  ILS  tasks  performed  during  the  acquisition  phase  and  then  con- 
stantly corrected  by  the  various  support  management  effectiveness  systems  utilized  by 
the  supply  system,  training  commands,  operations  commands,  systems  commands,  field 
support  activities,  etc.  The  support  system  is  set  up  for  long-term,  continuous  operation: 
therefore,  it  is  extremely  important  to  properly  plan  the  transitional  support  for  an  equip- 
ment. The  planning  requires  not  only  the  accurate  execution  ot  the  ILS  tasks  but  also 
ensuring  the  funds  and  other  resources  are  available  to  implement  the  ILS  recommenda- 
tions. Identifying  funds  requires  budgeting  actions  which  require  at  least  2 years  to  pro- 
duce the  allocation.  Identifying  personnel  resources  may  require  changing  manpower 
allocations,  recruiting  quotas,  setting  up  training  courses,  scheduling  a training  pipeline, 
and  so  forth  all  of  which  may  take  3 or  more  years  before  the  personnel  are  available. 
Obviously,  these  are  long  lead  times  which  may  exceed  the  time  needed  to  procure  the 
equipment:  this  puts  great  importance  on  the  phase-in  support  planning. 

All  too  often  the  planning  for  the  operations  phase  is  incomplete,  especially  in 
the  identification  of  funding  for  spare  parts  and  training.  When  the  new  system  is  put 
into  the  fleet  without  proper  planning  or  providing  the  initial  support,  it  looks  like  a 
mammoth  step  function  to  the  support  system;  the  support  system  “rings”  for  many 
years  trying  to  cope  with  the  perceived  discontinuity.  The  budget  cycle  ensures  that 
this  period  will  be  at  least  2 years  longer  than  the  equipment  phase-in  period;  this  may 
constitute  a significant  portion  of  a single.equipment’s  service  life.  The  unsupported 
equipment  suffers  abuses  that  often  permanently  degrade  performance  and  reliability: 
atrocious  availabilities  are  realized  (sometimes  under  I O'! ),  thus  cheating  the  user  out  of 
a needed  capability  and  expending  resources  (manpower  and  dollars)  better  spent  else- 
where. This  cycle  is  commonly  caused  by  paying  for  cost  overruns  in  the  acquisition 
phase  with  funds  initially  identified  for  support  procurement.  From  the  standpoint  of 
overall  damage  to  the  service,  it  is  far  better  to  reduce  the  number  of  equipments  pro- 
cured or  to  delay  their  introduction  until  support  is  made  available. 

When  personnel  training  is  indicated  and  supply  system  support  is  utilized  (almost 
always),  two  milestones  are  mandatory  in  the  acquisition  phase: 

• Training  Plans  Conference  convened  by  the  cognizant  systems  command  at 
least  3 years  prior  to  IOC  (table  VI 1 1- 1 ) 

• Provisioning  Conference  chaired  by  the  Ship  Parts  Control  Center  (SPCC)  at 
least  2 years  prior  to  IOC  (table  VI 1 1-2) 

These  conferences  reach  agreement  on  the  actions  to  be  taken  and  the  resources  required 
to  support  the  equipment.  When  the  acquisition  program  is  such  that  these  conferences 
can't  take  place  at  the  proper  time,  they  should  be  held  as  earls  as  possible,  and  the  pro- 
gram should  plan  and  budget  for  the  phase-in  support.  Where  feasible  and  cost-effective, 
long-term  warranty  service  provides  an  excellent  transition  vehicle.  The  equipment  may 
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TABLE  Vlll-I.  TRAINING  PLANS  CONFERENCE  PREREQUISITES 
(REF:  OPNAV  INST  1 500.8  SERIES). 

Operator  concept 

Maintenance  concept 

Skill  and  experience  level  estimates  especially  it  new  skills 
or  high  experience  levels  may  be  needed 

Estimate  of  manpower  requirements 

Plans  for  Fleet  introduction 

Training  requirements  estimate 

TABLE  VIII-:.  PROVISIONING  CONFERENCE  PREREQUISITES 
(REF:  NAVMAT  INST  4000.29  & MIL-STD-1 561 ). 

Support  concept 

Availability  requirements 

Risk  and  hazard  assessment 

Level  of  Repair/ Logistics  Support  Analysis  results 

Estimated  demand  requirements 

Essential  components  listing 

Long-lead-time  item  listing 

Standard  parts/parts-peculiar  requirements 

Plans  for  Fleet  introduction 

Tentative  support  management  plan 


mature  and  realize  reliability  growth  in  service  without  being  a maintenance  burden  to  the 
user  organization.  Contractor  depot  services  are  usually  implemented  after  the  warranty 
period  (3-0  years,  4 typically). 

A frequently  overlooked  support  requirement  is  the  technical  data  needed  to  set 
up  and  utilize  the  support  management  effectiveness  systems.  The  most  important  of 
these  systems  is  the  Maintenance  Data  Collection  System  (MDCS)  portion  of  the  Mainte- 
nance and  Material  Management  ( 3 M ) System.  MDCS  takes  maintenance  reports  from 
the  fleet  and  omputes  logistics  parameters  and  reliability,  maintainability,  and  availability 
(RM&A)  parameters.  The  logistics  parameters  are  straightforward  calculations  of  usage 
data  and  parts  demand  data  used  to  isolate  and  correct  supply  system  deficiencies.  The 
RM&A  parameters  are  used  to  isolate  latent  design  problems,  support  documentation 
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problems,  training  anil  manning  problems,  insufficient  test  equipment  allocations,  and  in- 
stallation problems  In  order  to  effect  the  discrimination  needed  to  isolate  such  problems, 
the  measured  computed  RM&A  values  must  be  compared  to  "should  perform”  values. 
Unfortunately,  the  required  values,  which  are  available  during  the  acquisition  cycle,  are 
frequently  not  made  available  to  the  logistics  coordinators  responsible  for  tracking  the 
equipment.  The  problems  are  further  exacerbated  by  the  failure  of  the  acquisition  com- 
munity to  coordinate  the  RM&A  parameters  specified  and  tested  in  the  acquisition 
phase  with  those  parameters  which  can  be  measured  in  the  support  phase  (see  chapter  IV, 
Conceptual  Phase)  and  by  the  failure  to  require  equipment  identification  reporting  codes 
to  be  nameplate  data,  to  provide  reporting  guidance  in  the  technical  manuals,  and  to  co- 
ordinate with  the  MIX’S  system  coordinators.  The  required  actions  are  straightforward 
and  simple  when  coordinating  actions  are  initiated  prior  to  the  equipment  qualification 
phase. 

The  most  overlooked  support  phase  is  phase-out.  Usually,  an  equipment  is  phased 
out  because  a replacement  equipment  is  being  phased  in.  It  scents  that  old  mission  re- 
quirements continue  forever,  although  they  may  be  integrated  into  new  missions.  As  an 
equipment  nears  the  end  of  its  service  life,  various  support  elements  become  uneconomic 
and  are  dropped.  Special  training  courses  are  dropped,  maintenance  contracts  are  termi- 
nated, etc.  Nobody  much  cares  except  the  poor  user  who  still  has  one  ot  the  old-timers. 
In  today’s  austere  budget  environment,  the  acquisition  program  for  the  replacement  equip- 
ment can  hardly  be  expected  to  assist  a lame  duck.  Most  ot  the  problems  which  can 
occur  during  phase-out  can  only  be  addressed  early  in  the  acquisition  cycle  by  establishing 
levels  of  repair  and  standardization  such  that  system-peculiar  items  are  minimal  and  are 
not  system  critical  and  minimal  skills  are  required  to  operate  and  maintain  the  system. 

A maintenance  contract  should  be  constructed  on  a cost-per-unit  basis.  Consideration  ol 
the  phase-out  problems  which  may  occur  for  an  equipment  being  replaced  by  the  current 
acquisition  may  make  it  desirable  to  alter  the  phase-in  rate  ot  the  new  equipment. 


PROJECT  I’HASE-OUT  PLANNING 

I'he  project  phase-out  plans  provide  lor  the  transition  ol  continuing  system  tasks  into 
functional  organizations  and  for  documenting  the  project  history.  The  phase-out  plan  should 
incorporate  the  project  WHS  and  should  annotate  each  task  as  completed  oi  continuing  tor 
recurring).  Completed  tasks  should  show  the  completion  date  and  the  person  or  organization 
responsible.  Continuing  or  recurring  tasks  should  show  who  was  responsible  in  the  project 
organization  and  the  person  or  organization  assuming  the  responsibility.  I he  coordinating 
plans  or  documents  and  any  pertinent  data  should  be  referenced,  and  storage  points  and 
holders  should  be  cited.  Common  documents  would  include  the  II  SP,  training  plans,  con- 
figuration baseline  and  data,  and  reprocurement  data  as  a minimum  with  many  other  docu- 
ments supporting  as  required.  I he  historical  section  should  include  a listing  ot  all  permanent 
project  documents,  reports,  and  data  from  beginning  to  end.  I liese  items  should  cover  the 
design  and  development,  testing,  acquisition.  II  S.  installation,  initial  field  data,  quality  and 
workmanship,  and  actual  costs  and  schedules  compared  to  the  planned  targets.  Any  significant 
achievements  of  the  project,  any  significant  problems,  and  any  lessons  learned  should  be 
incorporated  in  a narrative;  certainly  any  innovations  or  patents  should  be  mentioned.  I lie 
project's  successes  and  failures  provide  valuable  lessons  for  the  many  simil.n  projects  which 
will  muloubtedlv  follow  and  can  also  point  to  areas  in  which  organizational  policies  should 
be  improved. 
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MONITORING  THU  PERFORMANCE  OF  UQUIPMENTS  IN  THU  FLEET 


There  are  many  requirements  for  tracking  t he  performance  of  equipments  in  fleet 
use.  Operations  planners  need  to  know  which  ships  are  capable  of  performing  a particular 
mission,  support  planners  need  to  know  the  types  and  quantities  of  support  needed,  and 
acquisition  planners  need  to  know  how  past  acquisitions  are  performing.  There  are  two 
primary  systems  for  tracking  equipment  performance  the  CASRHPT  (Casualty  Report) 
system  and  the  Maintenance  Data  Collection  Subsystem  (MIX'S)  portion  of  the  Mainte- 
nance & Material  Management  <3M)  system.  The  two  systems  are  distinct  in  their  meth- 
ods and  purposes  of  reporting,  and  both  systems  serve  a multiplicity  of  purposes. 

The  CASRHPT  is  a message  report  sent  whenever  a failure  degrades  the  ability  of 
a ship  to  perform  its  missions.  Since  a CASRHPT  implies  the  ship  is  not  “ready  and 
willing"  to  take  on  any  task  assigned,  reports  are  frequently  not  made  unless  specifically 
required  by  the  operational  command  or  a mission  assignment  is  jeopardized  by  a failure. 
Vital  equipments  are  CASRHPT’d  for  a higher  percentage  of  failures  than  semivital  equip- 
ments and  semivital  more  than  nonvital.  No  amount  of  encouragement  has  seemed  to  in- 
fluence commanding  officers  to  CASRHPT  in  all  required  situations  because  of  the  ima- 
gined stigma  attached.  Failures  which  are  expected  to  be  corrected  within  a reasonable 
time  (usually  24  hours)  are  not  reported.  Besides  notifying  operations  planners  of  the 
ship's  reduced  capabilities,  the  CASRHPT  also  mobilizes  support  elements  to  assist  the 
ship.  There  are  three  basic  CASRHPT  categories  CASRHPT  for  parts,  CASRHPT  for 
outside  assistance,  and  CASRHPT  t > notify  the  chain  of  command  “my  equipment  is  down 
and  I'm  working  on  it."  The  first  two  types  are  by  far  the  most  prevalent.  Once  a 
CASRHPT  is  made,  status  reports  are  made  periodically  and  when  new  information  is 
available  until  the  CASRHPT  is  closed  out  by  a CASCOR  (casualty  corrected)  message. 

The  reporting  system  is  specified  by  NWIP-IO.  The  primary  benefits  of  CASRHPT  de- 
rive from  its  timeliness  of  reporting,  its  close  association  of  equipment  failures  to  ship 
capability,  and  its  ability  to  marshall  timely  support. 

The  CASRHPT  Master  Data  Bank  run  by  the  Fleet  Material  Support  Office.  Mech- 
anicsburg.  can  supply  historical  CASRHPT  records  by  equipment  for  the  last  3 years  con- 
taining the  severity,  reason  for  report,  time  to  repair,  parts  usage  data.  etc.  for  each  CAS- 
RHPT made.  Summary  data  are  also  available.  Also,  a Material  Condition  Index,  which 
is  a weighted  mathematical  factor  based  on  the  number  of  CASRHPTS.  severity,  and 
time  to  repair  for  an  equipment  over  a fixed  period,  is  useful  in  analyzing  trends  and 
the  impact  of  failures  on  fleet  operations  and  can  be  requested  from  the  data  bank.  A 
few  cautionary  notes:  time  to  repair  is  measured  in  calendar  time  rather  than  actual  ac- 
tive repair  time,  parts  usage  reflects  those  parts  thought  to  be  needed  at  the  time  of 
failure  and  may  not  be  representative  of  actual  failures,  and  the  CASRHPT  system  only 
receives  reports  of  significant  failures. 

CASRHPT  data  can  be  requested  through  (lie  cognizant  system  command  coordi- 
nating office  or  directly  from  1 MSO. 

The  MIX'S  records  all  nonpreventive-maintenance  actions  on  all  equipments  on 
the  CNO's  selected  equipment  list  or  in  the  Current  Ship's  Maintenance  Program  (CSMP) 
and  all  deferred  maintenance  actions.  There  is  constant  pressure  on  the  fleet  to  report 
thoroughly,  accurately,  and  in  a timely  manner:  as  a result,  virtually  all  (nonpreventive) 
maintenance  actions  are  reported  on  virtually  all  equipments  by  most  ships.  A great 
wealth  of  information  is  reported  through  MIX'S.  MIX’S  reporting  is  in  accordance  with 
OPNAV43-P2.  The  primary  benefits  of  MIX’S  arise  from  the  broad  range  of  detail  in- 
formation available  on  rnos|  fleet  equipments. 
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Tile  detailed  information  includes  grade  and  rate  of  the  reporting  technician,  man- 
hours  expended,  when  the  failure  was  discovered,  the  indication  of  trouble,  whether  the 
equipment  was  up,  down,  or  partially  down,  part  failure  data,  an  estimate  of  the  cause  of 
failure,  mean  down  time  (Ml)  I ),  the  causes  of  down  time,  mean  time  between  corrective 
maintenance  actions  (MTBCM).  mean  time  to  repair  (MTTR),  and  technician  narratives. 

This  information  is  collected  on  written  formats  from  each  ship,  key  punched  by  a 
coordinating  center  (one  on  each  coast),  and  entered  into  a central  data  bank  run  by  the 
Maintenance  Support  Office  Department  (MSOD)  of  I'MSO.  Reliability,  maintainability, 
and  availability  (RM&A)  parameters  are  derived  from  the  raw  data  by  the  Naval  Ship  en- 
gineering Center.  Norfolk  Division,  or  by  the  Ordnance  Maintenance  Management  Infor- 
mation Center.  NWS  Concord,  depending  on  the  type  of  equipment:  logistics  parameters 
are  derived  by  MSOD.  MIX'S  data  can  be  obtained  from  MSOD  or  through  the  cognizant 
system  command  coordinator. 

Both  systems  are  useful  in  performing  trend  analysis  on  equipment  problems. 
Maintenance  and  parts  usage  data  are  available  through  the  CASRI PT  system,  but  relia- 
bility data  are  not  available  other  than  what  can  be  inferred  from  CASRI  PI  frequency. 

Very  specific  maintenance  and  parts  usage  data  are  available  from  MIX'S:  additionally, 
reliability  and  equipment  usage  data  can  be  derived  with  MIX'S.  MIX'S  has  nearly  20 
times  more  reports  over  any  particular  period  than  CASRI  PT.  but  CASRI  Pis  are  ex- 
tremely timely  whereas  MIX’S  requires  5 to  0 months  to  collect  all  its  reports  for  a par- 
ticular month.  Both  systems  are  capable:  however,  most  RM&A  (racking  requirements 
are  best  met  by  MIX'S. 

Operations  planners  anil  support  planners  make  extensive  use  of  the  CASRI  PT 
and  MDCS:  however,  the  acquisition  community  has  traditionally  ignored  both  systems 
except  when  forced  to  support  “get  well”  programs  under  DARI  Ihis  situation  is  un- 
fortunate since  most  support  problems  are  created  within  the  acquisition  phase  and  many 
problems  can  only  be  remedied  by  the  acquisition  phase.  In  truth,  the  acquisition  com- 
munity has  demonstrated  its  lack  of  concern  for  the  equipment  or  its  user  once  the  pro- 
curement is  complete. 

A number  of  programs  are  run  by  the  support  community  utilizing  MDCS  data  to 
identify  parts  availability  problems,  training  problems,  and  equipment  reliability  problems. 
The  program  to  identify  equipment  reliability  problems  is  Project  Intercept.  Project  In- 
tercept tracks  key  equipments  and  identifies  potential  maintenance  and  logistics  problems 
by  measuring  four  RM&A  indicators  against  established  standards.  NAVMAT  has  recom- 
mended that  the  project  eventually  should  include  all  equipments  on  the  C NO  selected 
equipment  list.  Ihe  Intercept  indicators  are  MTBCM.  MTTR.  MDT.  and  A0.  The  MDT 
parameter  is  further  divided  into  MDT  (outside  assistance).  MDI  (parts),  and  MDT  (ships 
ops).  MDT  (parts)  is  used  to  determine  the  percent  of  parts  not  on  board  when  required. 
Safety -related  maintenance  actions  are  counted  also.  Parts  supply  problems  require  a re- 
sponse by  NAVSUPSYSCOM ; all  other  problems  require  a response  In  the  cognizant  sy  s- 
tems command  for  the  equipment.  The  standards  for  each  parameter  are  established  as 
a "level”  (should  perform)  and  "limit"  (worst  acceptable).  A0  has  a limit  set  by  NAV- 
MAT at  75  ; ; percent  parts  not  on  board  has  a limit  set  by  NAVSl'P  at  35','  (this  limit 
is  consistent  with  the  rules  governing  the  ship’s  parts  allowances)  I he  cognizant  syscom 
establishes  levels  and  limits  for  MTBCM.  MTTR,  and  gross  MDI  . The  system  would  work 
well  if  it  were  not  for  the  serious  problems  which  undermine  the  project 

The  most  obvious  problem  lies  in  the  fact  that  the  syscom  participants  in  Inter- 
cept are  the  3M  codes  within  the  logistics  directorates.  If  a problem  is  intercepted  which 
is  determined  to  require  design  changes,  some  developmental  effort,  or  other  major  engineering 


L 


Villa, 


w 


effort  lie.  beyond  the  capability  of  designated  field  maintenance  activities),  the  syscom  logis- 
tics code  is  unable  to  respond  since  it  does  not  have  cognizance  or  funding  in  these  areas. 
Rather,  the  syscom  acquisition  code  should  respond:  however,  the  acquisition  code  either 
responds  by  assigning  a low  priority  or  by  ignoring  the  problem.  I he  syscom  acquisition  code 
is  reinforced  in  this  posture  by  Ol’NAV  acquisition  priorities  and  b\  the  lack  of  "charter 
responsibilities"  for  "support  problems.” 

Ideally,  levels  and  limits  for  each  indicator  would  be  derived  for  the  operational 
requirements:  however,  tit  ■ sources  of  this  information  were  not  made  available  to  the 
logistics  codes  responsible  for  setting  the  standards.  Therefore,  levels  were  established 
by  measuring  a mean  value  over  a 2-year  period,  and  limits  were  established  at  the  dCf 
confidence  level  in  the  "bad"  side  of  the  level  standards.  This  methodology  is  incapable 
of  detecting  steadily  poor  performance. 

The  inconsistencies  between  the  way  RM&A  parameters  are  specified  and  the 
way  they  are  measured  require  that  the  raw  MIX'S  data  be  manipulated  to  produce  the 
indicators.  The  most  serious  problem  lies  in  the  derivation  of  MTBC'M.  which  also  af- 
fects the  calculation  ol  A0.  Since  very  few  equipments  are  time  metered,  equipment 
operating  time  must  be  estimated  from  ship  steaming  times.  On  some  equipments,  this 
methodology  is  no  doubt  quite  accurate;  however,  large  uncertainties  exist  for  many  e- 
quipments.  Equipment  operating  time  is  used  with  the  observed  failure  rate  to  calcu- 
late MTBC’M.  so  larg-  uncertainties  can  exist  in  MTBC’M.  This  is  not  quite  so  bad  as  it 
seems  since  the  same  criteria  are  applied  consistently  on  a given  equipment,  so  long- 
term trend  analysis  is  still  valuable.  However,  short-term  reports  and  the  comparison  of 
the  calculated  value  to  the  specified  value  of  reliability  are  virtually  meaningless  for  some 
equipments,  furthermore.  MTBC'M  is  used  to  calculate  A0  rather  than  MTBF.  I'll  is  is 
because  MIX’S  cannot  distinguish  between  a noncritical  failure  and  a “relevant”  failure. 

The  mechanism  is  available  in  the  MIX’S  to  make  this  distinction  through  the  “as  dis- 
covered status  code”:  however,  no  guidance  is  provided  to  the  reporting  technicians  in 
determining  what  constitutes  a "partial  failure”  and  what  constitutes  a "complete  fail- 
ure" in  a sense  that  will  provide  reporting  consistency  and  a meaningful  distinction  be- 
tween MTBC'M  and  MTBF  for  a given  equipment.  If  time  meters  were  provided  for  e- 
quipments  and  if  each  equipment  technical  manual  defined  partial  and  complete  fail- 
ures, these  major  problems  in  tracking  reliability  could  be  overcome  sufficiently  to  pro- 
duce results  with  a usable  confidence  level 

Reporting  accuracy  is  a most  important  issue  in  the  MIX'S.  In  the  past,  licet 
technicians  have  been  lax  in  their  reporting  because  of  the  time  required  to  fill  out  the 
report  form  with  its  multiplicity  of  codes  and  because  they  can  see  no  benefits  deriving 
from  this  effort.  More  recently,  training  courses  have  assisted  the  technicians  in  the  use 
of  3M  information  sources  and  have  touted  the  importance  of  reporting  accuracy,  burn 
more  important,  an  extreme  amount  of  pressure  has  been  applied  by  operational  com- 
manders through  inspections  to  cause  heightened  command  attention  to  3M.  Neverthe- 
less. the  conditions  under  which  reports  are  produced  sometimes  are  not  conducive  to 
reporting  accuracy.  This  accuracy  can  be  improved  if  the  equipment  reporting  codes 
(1IC.  API..  serial  number)  are  part  of  the  nameplate  information  and  if  the  equipment 
technical  manuals  define  areas  which  otherwise  cause  contusion  to  the  technician  (such 
as  the  distinction  between  a partial  and  a complete  failure). 

Ihe  selected  equipment  list  is  promulgated  by  ('NO  with  the  intention  of  re- 
ducing reporting  requirements  While  tins  is  a tenuous  premise  for  mans  ships,  care 
must  be  taken  to  avoid  not  tracking  significant  problems  and  potential  problems  1'hc 
present  method  seems  to  be  based  on  an  arbitrary  accept  reject  decision  made  for  each 
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equipment  nominated  by  the  syseoms.  TELCAM  recommends  that  the  selected  equipment 
list  include  all  vital  and  semivital  equipments,  all  new  equipments  for  at  least  4 years  after 
IOC  and  until  the  equipment  performance  consistently  tracks  at  acceptable  levels  based  on 
the  operational  requirements,  and  all  warranted  items. 

The  use  of  MIX'S  data  to  support  warranties  is  a natural  application  of  the  system. 
However,  the  problem  areas  discussed  above  must  be  resolved  by  prior  planning  for  an  equip- 
ment before  MIX’S  is  a valuable  warranty  tool.  An  additional  feature  to  the  system  is  to 
structure  a special  “deferred  action  taken”  code  say  "KW"  for  warranty  reporting  and 
tii  put  this  code  on  the  warranty  notice  on  the  equipment. 

If  information  is  desired  from  both  the  CASREPT  and  MIX'S  data  banks,  it  is  best  to 
forward  the  request  to  the  cognizant  system  command  coordinator  for  that  equipment.  If 
the  information  is  specifically  to  support  a syscom-sponsored  project,  the  request  should  be 
forwarded  through  the  sponsor.  At  NOSC,  Design  Engineering  Division  assists  in  formulating 
information  requests,  particularly  those  directly  to  MSOD  or  I MSO. 


UTILIZING  CONTRACTOR  SUPPORT 

There  are  two  levels  of  contractor  support  interim  and  long-term.  Interim  support 
is  frequently  used  during  the  phase-in  period  as  a stop-gap  while  operations  phase  support  is 
being  set  up.  Long-term  support  is  intended  for  the  entire  service  life  of  the  equipment. 

A contractor  may  not  be  particularly  well  prepared  to  provide  all  the  support  elements 
which  may  be  required,  especially  some  forms  of  training,  depot-type  maintenance,  and  test 
equipment:  requiring  these  elements  of  a contractor  will  cost  much  more  than  if  the  contrac- 
tor is  set  up  to  provide  them.  However,  the  supplier  of  the  equipment  is  always  the  best 
source  of  interim  support:  the  relative  inefficiencies  of  a contractor  in  some  services  can 
usually  be  tolerated  for  the  extent  of  a phase-in  period.  Although  the  contractor  may  not 
desire  to  be  held  responsible  for  interim  support,  there  is  actually  little  choice  in  most  cases. 
Contractors  not  familiar  with  warranty  clauses  will  tend  to  balk  at  their  application,  for 
instance.  1'he  contractual  clauses  requiring  interim  support  should  be  clear  and  concise.  Lite 
division  of  responsibility  between  the  government  and  the  contractor  should  be  clearly 
defined,  and  the  length  of  the  contractor’s  responsibilities  should  be  fixed.  Complex  contrac- 
tual requirements  will  tend  to  undermine  the  contractor’s  ability  to  plan,  estimate,  and 
execute  his  support  responsibilities. 

Long-term  support  brings  with  it  potentially  great  benefits  and  equally  great  pit  Calls. 

It  is  almost  always  economically  advantageous  to  make  use  of  a contractor's  commercial 
distribution  and  maintenance  system,  if  one  exists.  I here  are  innumerable  hidden  costs  in 
setting  up  any  support  system,  so  when  one  already  exists,  it  is  virtually  assured  that  it  will 
be  less  costly.  Equipment  which  has  been  developed  for  military  use  should  be  supportable 
by  the  government’s  facilities  if  the  equipment  has  been  properly  designed.  Equipments 
which  have  been  developed  for  a commercial  market  may  not  be  at  all  adaptable  to  the 
government’s  system:  therefore,  the  commercial  support  system  is  most  appropriate.  Commer- 
cial support  is  not  without  its  problems;  the  equipment  operational  requirement  must  be 
reviewed  to  ensure  that  the  existence  of  commercial  support  pitfalls  is  tolerable.  Some  of 
these  pitfalls  are: 

• Poorly  implemented  commercial  support 

• Labor  strikes 


• Financial  Failure  of  the  company 

• Susceptibility  to  disruption  during  conflict 


A poorly  implemented  support  system  should  be  discovered  during  a screen  o|  candidate 
equipments  (see  chapter  X I ).  Customer  comments  will  generally  reflect  the  competence  ot 
equipment  support.  A system  may  he  poorly  implemented  for  government  purposes  even 
though  it  is  excellent  for  a particular  market.  For  instance,  a market  confined  to  the  Great 
I akes  region  might  be  serviced  quite  well  from  Cleveland,  but  the  service  would  be  deplorable 
for  Diego  Garcia.  An  economic  analysis  of  the  pipeline  assets  required  tor  adequate  support 
may  be  necessary  to  resolve  this  problem.  Obviously,  some  requirements,  especially  vital  sys- 
tems. do  not  lend  themselves  to  nonorganic  support,  much  less  contractor  support.  However, 
many  requirements  could  tolerate  the  relatively  short  delays  that  some  commercial  worldwide 
support  systems  can  provide.  These  systems  may  remain  intact  even  during  a major  conflict, 
but  use  of  their  support  services  may  be  disrupted.  An  enemy  would  not  call  a truce  to 
allow  you  to  take  your  broken  gidget  to  the  authorized  factory  service  center  in  his  occupied 
territory,  nor  could  you  depend  on  a service  center  run  by  belligerent  partisans.  I hese  factors 
must  be  taken  into  account  in  the  support  planning:  generally,  these  problems  should  be 
resolved  by  accepting  or  rejecting  a candidate  equipment  on  its  supportability  criteria. 

Any  support  system  could  conceivably  be  affected  by  a labor  strike,  although  govern- 
ment facilities  should  be  considerably  less  susceptible.  Strike  tolerability  is  a function  of  I he 
criticality  of  the  equipment.  Nonvital  equipments  can  tolerate  long  shutdowns  of  some 
support  elements  while  vital  equipments  cannot.  Several  options  are  available  to  diminish 
the  impact  of  a strike: 

1.  An  injunction  can  be  imposed  for  reasons  of  national  security. 

2.  Additional  pipeline  assets  can  be  required. 

.A  A secondary  service  agreement  can  be  employed. 

4.  The  government  can  take  over  support  of  the  item. 

An  injunction  can  be  employed  only  in  rare  cases  during  peacetime  because  of  its 
political  effects;  it  is  primarily  a wartime  tool.  Additional  pipeline  assests  can  normallv  be 
made  available  only  by  very-high-volume  industries,  since  the  assets  come  out  of  inventory. 
Low-volume  industries  which  are  subject  to  frequent  labor  problems  may  warrant  a greater 
initial  purchase  in  order  to  provide  reserve  pipeline  assets;  this  is  subject  to  the  scrutiny  of  an 
economic  analysis.  Secondary  service  agreements  are  frequently  employed  in  commercial 
practice:  in  this  case,  a second  company  takes  over  the  support  load  when  the  primary  system 
cannot  meet  its  demands.  The  existence  of  such  agreements  should  be  investigated  during  the 
screening  process;  if  none  exists,  a requirement  that  such  an  agreement  be  created  might  be 
negotiated  into  the  contract  provided  that  the  government's  business  is  sufficiently  large  to 
contain  the  necessary  economic  incentives.  Highly  proprietary  items  usually  cannot  emplov 
blanket  secondary  service  agreements  independent  of  the  parent  company:  however,  systems 
of  independent  factory-authorized  serv  ice  representatives  are  very  effective.  Instances  m 
which  the  government  might  take  over  support  of  its  items  are  limited  because  ol  the  invest- 
ment required.  A notable  illustration  of  such  a case  involved  the  overhaul  of  I’olar ns  guidance 
sections,  which  was  originally  performed  by  the  contractor.  An  18-month  strike  threatened 
to  cripple  this  vital  system,  so  the  government  took  over  the  overhaul  responsibilities.  I here 
are  a number  of  arrangements  whereby  the  government  becomes  a secondary  service  agent 
for  a company.  Normally,  only  a fraction  of  the  government's  support  requirements  are  met 
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by  government  facilities,  but  some  protection  is  provided  at  the  expense  of  the  cost  of  main- 
taining the  capability.  Some  savings  are  still  realized  over  full  government  support. 

The  financial  collapse  of  the  contractor  could  also  destroy  the  support  system  for  his 
equipment.  Secondary  service  agreements  are  the  most  effective  protection  against  this  , 
problem.  However,  equipments  containing  sole  source  parts-peculiar  can  defy  virtually  any 
protective  ploy.  The  government  can  require  the  posting  of  a surety  bond,  but  this  only 
provides  monetary  compensation  for  loss.  Meanwhile,  the  support  of  the  equipment  is  still 
nonexistent.  The  government  can  also  require  that  all  product  and  process  documentation  be 
turned  over  in  event  of  contractor  failure.  In  theory,  the  government  can  then  set  up  its  own 
support:  in  practice,  the  documentation  is  seldom  adequate  to  allow  support  to  be  reestab- 
lished. In  the  final  analysis,  the  only  solution  proved  practical  is  the  second  service  agreement, 
short  of  government  rescue  of  the  company. 

There  is  a legal  problem  with  second  service  agreements  between  a contractor  and  the 
government  if  an  equipment  contains  proprietary  parts  or  designs.  The  government  can  be- 
come liable  to  damages  if  it  discloses  the  proprietary  information.  Even  when  the  govern- 
ment is  faultless,  it  may  still  become  embroiled  in  protracted  legal  proceedings.  In  such  cases, 
the  government  has  been  found  at  fault  for  not  taking  adequate  measures  to  protect  the 
information. 
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PART  15:  KEY  DISCIPLINES 


IX.  LIFE-CYCLE  COSTING  (LCC) 

In  system  development  programs,  cost  is  a major  consideration  in  decisions  in- 
volving selection  of  candidate  systems  or  design  alternatives.  Therefore,  the  cost  concept 
and  the  methodology  of  cost  analysis  to  be  chosen  and  used  in  support  of  such  decision 
making  are  of  basic  importance.  In  the  past,  cost  considerations  in  system  acquisition 
were  generally  narrow  in  perspective,  with  emphasis  being  limited  primarily  to  the  area  of 
hardware  development  and  procurement,  while  deployment  costs  associated  with  system 
operation  and  maintenance  and  support  were  given  little  attention  or  completely  ignored. 
This  traditional  cost  approach  often  resulted  in  equipments  that  were  costly  to  maintain 
and  support. 

An  alternative  approach  is  to  consider  system  life-cycle  costs  as  a basis  for  deci- 
sion making.  Based  on  the  total-cost  concept,  the  life-cycle  costing  technique  can  be  a 
useful  analysis  tool  and  a deeision-making  aid  throughout  the  system  development  proc- 
ess. As  a basic-  advantage  of  using  this  technique,  visibility  is  ensured  for  all  major  com- 
ponents of  the  system  total  cost.  This  permits  decisions  to  be  made  on  the  basis  of  the 
complete  system,  and  allows  attention  to  be  focused  on  those  system  parameters  and 
support  factors  with  influence  on  system  operating  costs  as  well  as  hardware  acquisition 
costs.  However,  implementation  of  life-cycle  costing  implies  certain  requirements,  such 
as  basic  cost  data  and  support  expertise.  In  addition,  the  decision  maker  or  project  man- 
ager must  be  familiar  with  the  underlying  concept  and  methodology  involved. 

The  basic  elements  and  applications  of  life-cycle  costing  are  addressed  in  this 
chapter,  the  purpose  of  which  is  to  provide  the  project  manager  with  an  understanding 
of  life-cycle  costing  and  its  applications  in  system  development. 

SYSTEM  LIFE-CYCLE  COSTS 

System  total  life-cycle  cost  is  composed  of  many  elements  representing  the  var- 
ious resources  which  are  required  for  system  acquisition  and  operation  throughout  the 
entire  life  of  the  system.  In  the  order  of  their  occurrence,  the  different  system  cost 
elements  may  be  separated  into  the  following  categories: 

• Research  and  development.  Costs  primarily  associated  with  the  development 
of  a r.ew  system  or  capacity  to  the  point  at  which  it  is  ready  for  procure- 
ment and  operational  use 

• Investment.  Costs  beyond  the  development  phase  to  introduce  a new  sys- 
tem or  capability  into  operational  use.  including  installation  and  checkout 

• Operation  and  support  (O&S).  Recurring  costs  of  operation,  maintenance, 
and  logistics  support  of  the  system 

The  purpose  of  identifying  and  displaying  system  costs  as  separate  categories  is 
to  facilitate  their  evaluation  by  the  decision  maker  or  planner,  since  the  costs  associated 
with  each  phase  of  the  system  life  cycle  bear  a different  relationship  to  time  and  to  the 
number  of  units  of  the  system  to  be  procured.  R&l)  costs  are  relatively  independent 
of  both  time  and  the  number  of  system  units  to  be  procured.  Investment  costs  are  also 
independent  of  time,  but  are  directly  related  to  the  number  of  system  units  to  he 
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deployed  and  to  the  expected  service  life  of  the  system.  Because  of  their  relative  indepen- 
dence of  time.  R&D  and  investment  costs  are  considered  as  one-time  costs.  By  contrast,  re- 
current O&S  costs  are  long-term  costs  which,  for  systems  witli  long  service  lives,  may  ac- 
count for  the  major  part  of  the  system  total  life-cycle  cost. 

COST  ESTIMATING  METHODS 

Besides  identification  of  system  cost  elements,  the  major  effort  of  life-cycle  cost- 
ing is  the  development  of  cost  estimates.  The  means  for  making  cost  estimates  are  many 
and  varied,  ranging  from  the  use  of  expert  opinions  and  informed  judgment  as  the  only 
basis  for  an  estimate  to  the  application  of  more  formal  methods.  The  more  formal  meth- 
ods used  are  generally  of  three  types  statistical,  engineering,  and  accounting. 

The  statistical  method  of  cost  estimating  is  based  on  the  use  of  historical  cost 
data  of  past  or  existing  systems  and  the  application  of  selected  statistical  techniques,  such 
as  multiple  regression  and  correlation  analysis  and  scatter  diagrams.  By  use  ot  the  statis- 
tical techniques  the  historical  data  are  analyzed  to  find  functional  relationships  between 
system  costs  and  specific  elements  of  system  description,  such  as  weight,  speed,  activity 
rates,  and  number  of  personnel.  The  derived  relationships,  often  referred  to  as  cost  es- 
timating relationships  (CERs).  are  used  to  estimate  or  project  costs  for  new  systems.  It 
is  assumed  that  the  historical  data  used  are  valid  if  the  systems  which  generated  such  data 
are  sufficiently  similar  to  the  new  systems  under  consideration.  This  type  of  cost  esti- 
mating method  is  usually  applied  at  relatively  high  levels  of  aggregation  for  cost  studies 
of  advanced  systems,  particularly  during  the  early  development  stages  of  the  system. 

The  essence  of  the  engineering  method  of  cost  estimating  is  to  successively  break 
down  the  system  or  item  of  hardware  into  lower-level  components  until  meaningful  con- 
jectures about  the  cost  implications  of  the  various  components  can  be  made.  CERs  are 
often  applied  at  this  lower  level  of  detail.  The  resulting  estimates  lor  the  individual  com- 
ponents, together  with  the  cost  estimates  for  integrating  the  components,  are  then  com- 
bined to  obtain  the  total  estimate.  One  of  the  useful  features  of  the  engineering  method 
is  that  it  helps  to  separate  those  parts  of  the  system  or  problem  which  require  novel 
treatment  from  those  which  can  be  dealt  with  by  conventional  means.  However,  a dis- 
advantage of  this  method  is  that  it  frequently  leads  to  underestimating  because  inadequate 
allowance  is  made  for  the  cost  of  integration.  Therefore,  when  the  engineering  method 
is  used,  the  statistical  method  is  often  also  applied  at  a higher  level  of  aggregation  to  en- 
sure against  underestimating. 

The  accounting  method  relies  on  the  fact  that  certain  factors  or  estimating  rela- 
tionships are  inherent  in  the  books  ol  account,  financially  or  otherwise.  Overhead  rates, 
labor  rates,  and  material  consumption  rates  are  examples.  Idle  method  is  conceptually 
simple,  but  it  does  usually  require  that  estimates  be  made  at  a relatively  lower  level  ol 
detail  than  is  generally  practical.  Furthermore,  when  the  accounting  method  is  used,  it 
is  necessary  to  exercise  extreme  caution  so  that  misleading  impressions  arising  from  using 
the  relationship  out  of  context  are  not  conveyed. 

The  different  cost  estimating  methods  may  be  applied  singly  or  in  combination, 
depending  on  the  given  problem.  Selecting  the  method  or  methods  to  be  used  is  depen- 
dent on  many  factors,  including  time  available  to  make  the  estimate;  level  ol  detail  and 
preciseness  needed;  type  of  analysis  to  he  performed;  availability  of  system  descriptive 
information;  lorm  and  availability  of  relevant  historical  data;  and  extent  ot  departure  trom 
experience  of  the  system  for  which  cost  estimates  are  to  be  obtained.  Ultimately,  the  final 
choise  is  dependent  on  the  experience  and  preference  of  the  individual  cost  analy  st 
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SYSTEM  COST  ANALYSIS 


There  are  two  types  of  system  cost  analysis  which  are  particularly  applicable  to 
decision  making  in  system  development  “intrasystem”  and  “intersystem.”  The  first 
involves  the  comparison  of  different  designs  for  a given  system,  while  the  second  involves 
the  comparison  of  a set  of  competing  candidate  systems. 

In  “intrasystem”  analysis,  the  emphasis  is  on  the  selection  of  the  configuration 
or  characteristics  of  a system.  Costs  arc  used  to  assist  in  the  selection.  System  character- 
istics are  sought  which  provide  the  minimum  cost  for  various  performance  levels  or.  con- 
versely, costs  are  used  to  indicate  those  achievable  characteristics  which  maximize  per- 
formance for  various  possible  funding  levels.  This  type  of  analysis  can  be  useful  in  eval- 
uating the  cost  impacts  of  alternative  design  specifications  and  system  operating  character- 
istics. It  provides  an  in-depth  consideration  of  a single  system,  anil  may  be  used  to  help 
put  together  an  initial  system  description. 

In  “intersystem”  analysis,  the  emphasis  is  on  comparing  two  or  more  systems  com- 
peting for  the  same  mission.  Effort  is  concentrated  on  isolating  those  features  of  each  ot 
the  competing  systems  that  cause  costs  to  differ.  It  is  assumed  that  each  of  the  competing 
systems  has  been  optimized,  or  suboptimized,  as  to  configuration  so  that  the  comparison 
is  meaningful,  or  "fair.”  This  type  of  analysis  is  applicable  for  selecting  the  preferred  sys- 
tem, and  is  often  applied  during  the  conceptual  phase  of  the  system  life  cycle. 


RELIABILITY.  MAINTAINABILITY.  AND  SUPPORT  FACTORS 

Traditionally,  consideration  of  reliability  and  maintainability  during  system  devel- 
opment is  concerned  primarily  with  meeting  operational  requirements  rather  than  reducing 
system  support  costs.  As  a result,  there  is  often  a tendency  to  specify  minimum  levels 
of  reliability  and  maintainability  that  will  satisfy  the  system  availability  requirement  with- 
out considering  the  effects  on  the  life-cycle  cost  of  system  support.  On  the  other  hand,  it 
has  been  generally  indicated  by  results  of  studies  on  past  and  existing  systems  that  higher 
levels  of  reliability  and  maintainability  are  warranted  in  order  to  reduce  the  high  cost  of 
system  maintenance  and  logistic  support  which,  for  systems  with  long  service  I i fe  spans. 
is  the  predominant  component  of  system  total  life-cycle  cost. 

A list  of  elements  associated  with  the  cost  of  system  support  is  shown  in  figure 
IX- 1 . together  with  the  principal  factors  which  contribute  to  mission  readiness.  Note  that 
the  support  elements  are  not  independent  of  each  other,  but  are  defined  by  the  total  sup- 
port plan.  Each  element  must  be  properly  related  with  every  other  element  in  order  to 
provide  adequate  maintenance  and  logistic  support  for  the  system  equipments  over  the  life 
cycle  of  the  system.  If  the  definition  or  scope  of  one  of  the  elements  is  altered,  changes 
are  induced  in  others  which  may  significantly  affect  the  coherence  of  the  support  system 
and  cause  redundancy,  mismatch,  or  omission  of  vital  factors.  Such  changes  may  have 
impact  not  only  upon  the  effectiveness  of  system  support  but  also  upon  the  total  life- 
cycle  cost  of  the  system. 

The  effects  of  reliability  and  maintainability  on  support  cost  can  be  seen  by  con- 
sidering the  largest  item  in  the  support  cost  package  the  cost  of  maintenance,  which 
is  the  combined  cost  of  maintenance  personnel  and  spare  parts  used  for  repair.  The  num- 
ber of  spare  parts  used  for  repair  is  directly  related  to  the  number  of  equipment  failures 
which  occur  during  a given  period,  while  the  man-hours  needed  for  corrective  maintenance 
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Figure  IX- 1 System  support  in  life-cycle  cost. 
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is  related  to  both  the  number  of  failures  during  the  period  and  the  mean  time  to  re- 
pair (MTTR).  which  is  a measure  of  maintainability.  Since  the  number  of  equipment 
failures  occurring  during  any  time  period  is  a function  of  equipment  reliability,  or  the 
mean  time  between  failures  (MTBF),  it  follows  that  the  costs  of  maintenance  personnel 
and  spare  parts  will  be  high  for  low  reliability  and  maintainability.  Furthermore,  a low- 
reliability  system  will  also,  over  the  life  cycle,  require  more  test  equipment,  technical 
manuals,  and  other  support  items  such  as  facilities  and  spare  parts  transportation. 

Thus,  reliability  affects  all  support  cost  elements,  directly  or  indirectly.  Main- 
tainability is  important  for  similar  reasons.  In  general,  it  may  be  shown  that  the  system 
acquisition  costs  (R&D  and  investment)  increase  with  reliability,  while  the  recurrent  costs 
of  operation  and  support  decrease.  These  relationships  am  shown  in  figure  IX-2.  • 

Maintenance  and  support  planning  is  another  area  with  important  influence  on 
system  total  life-cycle  cost.  Its  function  is  to  provide  a total  support  program  for  the 
system  being  designed  and  developed.  Ideally,  the  total  support  program  should  be  well 
balanced  and  tailored  for  the  system  under  development  in  the  interest  of  efficiency  and 
economy.  However,  the  impact  of  maintenance  and  support  planning  on  support  cost 
is  felt  far  upstream  from  equipment  delivery.  Therefore,  the  maintenance  and  support 
effort  should  be  started  early  in  the  development  program  if  it  is  to  have  an  opportunity 
to  explore  the  various  concepts  for  supporting  the  system  and  to  influence  design  deci- 
sions in  system  maintenance  and  support  features.  System  support  cost  must  also  be 
emphasized  early  and  must  be  included  as  a basis  for  evaluating  and  selecting  the  main- 
tenance and  support  alternatives  during  the  various  planning  stages,  while  cost  analysis 
and  tradeoff  (such  as  level  of  repair  analysis)  should  be  performed  to  help  establish  the 
repair  strategies  and  logistics  support  requirements.  As  the  system  development  pro- 
gresses and  information  necessary  for  spelling  out  the  support  program  becomes  avail- 
able. the  maintenance  and  support  plans  should  be  appropriately  defined  at  each  stage. 

This  will  provide  a base  from  which  life-cycle  cost  of  system  support  can  be  evaluated 
for  use  in  program  decisions. 

SYSTEM  DEVELOPMENT 

System  development  is  a process  involving  a series  of  engineering  activities  and 
management  decisions  the  aim  of  which  is  the  successful  selection  and  specification  of  a 
preferred  system  design  for  procurement  and  production.  System  life-cycle  costs  must 
be  a part  of  the  basis  of  decision  making  involving  the  selection  of  system  candidates  or 
design  alternatives.  The  decision  maker  must  take  into  account  the  future  cost  impli- 
cations of  system  operation  and  support  as  well  as  the  near-term  costs  of  hardware  pro- 
curement and  installation.  The  primary  purpose  of  life-cycle  costing  is  to  help  define  and 
describe  these  cost  implications. 


CONCEPTUAL  PHASE 

Hie  conceptual  phase  consists  of  many  activities,  including  identification  and  de- 
finition of  conceptual  systems:  technical  feasibility  and  tradeoff  studies:  and  experimenta- 
tion and  test  of  operational  requirements,  key  components,  and  critical  subsy  stems  The 
output  ol  this  phase  includes  a set  of  alternative  technical  approaches  or  candidate  systems 
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Figure  IX-2.  Lifecycle  cost  vs  reliability  (Relationships  for  maintainability  are  similar.) 
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As  indicated  in  figure  I.X-.>.  the  role  of  life-cycle  costing:  during  the  conceptual 
phase  is  to  assist  in  the  selection  of  the  preferred  system.  Depending  on  the  type  of 
system  development  involved,  a candidate  system  may  be  evaluated  according  to  pre- 
established  criteria  (such  as  design-to-cost  goals),  or  its  cost  performance  may  be  com- 
pared with  that  of  an  existing  system  that  is  to  be  replaced.  Since  systems  in  the  early 
stages  of  the  system  life  cycle  are  described  in  terms  of  broad  performance  parameters 
and  concepts,  cost  estimates  of  the  candidate  systems  will  be  provided  at  relatively  high 
levels  of  aggregation.  Such  cost  estimates  are  usually  obtained  from  empirically  derived 
cost  estimating  relationships;  that  is.  by  statistical  methods.  The  main  purpose  of  the 
estimates  during  this  phase  is  to  detect  cost  differences  among  the  candidate  systems. 

The  emphasis  is  on  indicating  the  cost  impacts  of  the  major  system  features  rather  than 
on  the  detail  elements  of  the  individual  systems. 

The  type  of  analysis  used  for  comparing  the  candidate  systems  and  their  costs 
depends  on  whether  other  factors  besides  cost,  such  as  system  effectiveness  and  perfor- 
mance. are  also  included  as  criteria  for  the  preferred  system  selection.  Some  form  of 
cost-effectiveness  analysis  will  be  indicated,  for  example,  if  system  effectiveness  is  a 
consideration  and  can  be  meaningfully  quantified.  For  cases  in  which  emphasis  is  on 
minimizing  cost  rather  than  maximizing  performance  (or  effectiveness),  a method  for 
comparing  the  candidate  systems  and  selecting  the  preferred  system  is  simply  to  fix  or 
specify  the  level  of  system  performance  required  and  identify  the  system  with  the  lowest 
cost  among  the  alternatives.  The  system  identified  is  the  minimum-cost  solution  for  the 
specified  level  of  performance.  It  is  assumed,  in  this  type  of  system  comparison,  that 
any  extra  performance  possessed  by  an  individual  candidate  above  the  level  required  is 
of  little  value  and,  therefore,  does  not  warrant  extra  cost. 

Since  the  stress  is  on  comparing  system  costs,  application  of  life-cycle  costing 
during  the  conceptual  phase  depends  on  whether  alternatives  are  available.  In  addition, 
since  cost  estimates  during  the  early  stages  of  the  system  life  cycle  are  usually  coarse, 
application  of  life-cycle  costing  during  the  conceptual  phase  is  most  meaningful  when 
there  are  significant  differences  among  alternative  systems.  In  any  event,  life-cycle  costs 
should  be  considered  as  early  as  possible  for  maximum  influence  on  the  system  and  the 
associated  costs  of  operation  and  support. 


VALIDATION  PHASl 

The  validation  phase  begins  with  system  requirements  established  in  the  concep- 
tual phase  (fig  IX-I  ).  examines  different  design  concepts  and  approaches,  and  identifies 
a cost-effective  system  design  for  the  following  engineering  effort.  Ihe  validation  phase 
is  also  concerned  with  sy  stem  maintenance  and  support  concept  alternatives  as  .well  as 
system  operating  modes.  The  outputs  include  the  selected  system  design  specifications 
(development  specification)  and  the  technical  and  program  plans  for  the  next  phase. 

Specifically,  life-cycle  costing  may  be  applied  during  the  validation  phase  to  as- 
sist in  the  following 

• ( valuation  and  selection  ol  system  designs 

• Design  analysis  and  tradeoff 

• (-'valuation  and  selection  of  maintenance  and  support  concepts 
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Figure  IX-3.  Life-cycle  costing  in  system  development 
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• 1'vuluation  of  cost  impacts  of  parameter  changes  (such  as  reliability  and  main- 
tainability) 

• Assessing  cost  implications  of  equipment  selection  options  (commercial  vs  mili- 
tate equipment,  builil  vs  buy  or  modify,  etc) 

Application  of  life-cycle  costing  during  validation  is  also  useful  in  evaluating  and 
selecting  maintenance  and  support  concepts.  I his  is  particularly  important  since  this  is 
the  stage  at  which  system  maintenance  and  support  concepts  are  initially  defined  and 
considered.  Note,  however,  that  as  more  detailed  elements  of  the  system  arc  being  con- 
sidered. the  life-cycle  costing  effort  required  during  the  validation  phase  is  expected  to  be 
much  greater  than  that  required  for  the  previous  phase.  Therefore,  a life-cycle  costing 
model  is  generally  necessary  in  order  to  reduce  the  cost  estimating  burden  as  well  as  to 
ensure  the  availability  of  timely  results. 


FULL-SCALE  DEVELOPMENT 

The  full-scale  development  phase  is  the  intensive  engineering  phase  during  which 
the  system,  including  all  the  items  necessary  for  its  support,  is  designed,  fabricated,  and 
tested.  The  output  is  a model  of  the  system  configuration  which  is  demonstrated  and 
evaluated  to  meet  all  requirements  based  on  the  specifications  generated  during  valida- 
tion. The  chosen  system  design  concept  is  examined  in  detail,  and  changes  or  modifi- 
cations necessary  to  complete  and  optimize  the  system  configuration  arc  introduced. 

The  original  design  concept  may  even  be  replaced,  if  its  pursuit  proves  to  be  undesirable 
or  if  it  is  found  to  have  serious  shortcomings  as  it  is  refined  during  the  engineering  stage. 
The  role  of  life-cycle  costing  during  this  phase  is  primarily  to  help  evaluate  such  major 
design  changes  by  assessing  their  cost  impacts,  particularly  in  the  areas  of  maintenance 
and  support.  Life-cycle  costing  may  also  be  input  to  decisions  such  as  the  choice  ol 
standard  or  nonstandard  parts.  Since  engineering  design  entails  many  changes,  it  is  ap- 
parent that  there  are  many  areas  involving  a choice  among  alternatives  where  cost  is  a 
consideration.  However,  the  emphasis  in  life-cycle  costing  must  be  on  major  items  or 
areas,  as  it  is  not  practical  to  optimize  every  detail  element  ol  the  system. 


ADDITIONAL  AREAS  OF  APPLICATION 

Two  additional  areas  for  possible  application  of  life-cycle  costing  are  ( I ) source 
selection  and  (2)  contractual  commitments  in  material  procurement  and  system  ac- 
quisition lhey  arc  covered  by  the  following  DoD  documents  (tig  1X-4) 

• I itc  Cycle  Costing  Procurement  (iuide  (I.CC-I  I 

• Casebook,  l ife  Cycle  Costing  in  Equipment  Procurement  tl.CC-2) 

• 1 ale  Cycle  Costing  (iuide  for  System  Acquisitions  (I  CC-.D 

I.CC-I  addresses  life-cycle  costing  in  the  procurement  ol  material  and  hardware 
other  than  complete  weapon  systems  It  presents  a set  ot  general  guidelines  which  may 
be  modified  to  suit  the  needs  of  a specific  application  I samples  ol  applications  ot  the 
general  guidelines  arc  given  m ICC-2  I ( ( addresses  lile-cycle  costing  in  the  acquisition 
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LIFE  CYCLE  COSTING  PROCUREMENT  GUIDE  (INTERIM).  JULY  1970 

LIFE  CYCLE  COSTING  GUIDE  FOR  SYSTEM  ACQUISITIONS  (INTERIM),  DOD,  JANUARY  1973 


f igure  IX -4  Life-cycle  costing  reference  documentation. 


of  material  at  the  complete  system  level.  It  covers  the  application  of  the  life-cycle  cost 
concept  to  the  various  acquisition  strategies  and  contains  an  operating  and  support  cost 
model  for  predicting  system  ownership  cost. 


IMPLEMENTING  LIFE-CYCLE  COSTING 

Certain  basic  requirements  or  preparatory  steps  which  will  help  to  assure  useful 
results  from  the  application  of  life-cycle  costing  are  described  below. 


PROBLEM  DEFINITION 

Defining  the  problem  is  the  first  step  in  a life-cycle  cost  analysis.  During  the 
problem  definition  phase,  close  cooperation  between  the  cost  analyst  and  the  system  en- 
gineer is  required  so  that  the  system  to  be  studied  can  be  adequately  described  for  cost- 
ing purposes.  Before  the  analysis  effort  can  begin,  the  cost  analyst  must  have  ( 1 ) a des- 
cription of  the  system  to  be  costed  and  (2)  cost  ground  rules  for  the  particular  study. 

As  part  of  the  system  description,  all  relevant  cost-significant  elements  including  sys- 
tem performance  characteristics,  physical  characteristics,  and  operational  assumptions 
must  be  identified  and  described.  Since  the  system  descriptions  needed  by  the  cost  ana- 
lyst can  differ  considerably  from  those  used  by  the  system  engineer,  direct  communication 
between  the  cost  analyst  and  the  system  engineer  is  usually  necessary.  A complete  un- 
derstanding of  the  ground  rules  is  also  necessary.  Examples  include  the  type  of  cost 
index  to  be  used  (eg.  15-year  system  cost):  rules  regarding  amortization  or  discounting; 
date  for  which  all  prior  costs  are  to  be  considered  as  sunk  costs:  and  special  rules  re- 
garding base  pay  of  operating  personnel  and  attrition  rates. 


DATA  ACQUISITION 

The  types  of  data  needed  are  identified  from  the  system  cost  elements  list  which 
is  based  on  the  system  description.  The  data  acquisition  effort  must  begin  long  before 
the  data  are  needed,  and  the  data  must  cover  the  needs  for  specific  systems,  including 
the  types  of  resources,  cost  equations,  and  cost  factors.  The  data  collected  should  be  or- 
ganized. with  means  lor  indexing  and  classifying  cost  and  related  data  and  means  tor 
ready  access  to  the  data  by  users. 


DERIVATION  OF  ESTIMATE 

This  is  the  step  in  system  costing  which  is  concerned  with  the  actual  calculation 
of  the  cost  estimates.  The  cost  estimates  are  developed  within  the  framework  of  cost 
element  lists,  the  purpose  of  which  is  to  identify  and  account  for  all  elements  of  cost 
associated  with  the  system.  The  cost  elements  are  subdivisions  of  the  major  cost  cate- 
gories. including  development,  investment,  and  operation  and  support  costs  There  is 
no  single  cost  element  list  which  can  serve  the  needs  of  all  problems  1 ists  have  to  be 
flexible  in  their  makeup,  as  they  must  be  adapted  to  the  type  of  system,  the  nature  of 
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the  problem,  and  the  type  of  analysis  considered.  The  list  should  highlight  the  key 
features  of  the  system  and  permit  maximum  use  of  the  data  collected.  Once  a satis- 
factory cost  element  list  is  developed,  calculation  of  cost  estimates  is  accomplished 
through  cost  estimating  relationships,  cost  models,  or  other  means. 


PRESENTATION  OF  ESTIMATES 

This  step  is  concerned  with  the  communication  of  results  to  the  user,  and  it  is 
related  to  the  first  step,  problem  definition.  For  ease  of  understanding,  the  cost  esti- 
mates must  be  presented  in  terms  and  formats  as  well  as  at  a level  of  detail  appropriate 
for  the  type  of  decision  to  be  made.  The  terms  and  formats  vary  with  the  application, 
and  they  should  be  given  early  consideration. 


ANALYSIS  DOCUMENTATION 

Proper  documentation  of  the  cost  analysis  is  important  to  both  the  analyst  and 
the  users  of  the  analysis.  In  addition  to  its  use  for  review  and  evaluation,  the  documen- 
tation may  serve  as  a record  of  the  study  and  a source  of  data  which  can  be  used  for 
other  studies.  For  review  and  evaluation  purposes,  the  documentation  should  clearly 
describe  and  discuss  the  procedures  and  data,  as  well  as  the  data  sources  used.  It  should 
also  permit  the  cost  estimates  to  be  reproduced,  if  necessary,  via  the  procedure  or  proc- 
ess described. 


REVIEW  AND  EVALUATION 

The  user  has  the  critical  task  of  judging  the  cost  estimates  and  evaluating  them 
as  to  suitability  and  credibility.  Since  direct  verification  for  accuracy  is  impractical, 
particularly  when  cost  estimates  are  used  for  decisions  involving  systems  far  into  the 
future,  emphasis  must  be  placed  on  evaluating  the  validity  of  the  cost  study  itself  and 
the  analysis  underlying  it.  In  particular,  areas  such  as  data,  data  sources,  methods,  pro- 
cedures. and  conclusions  should  be  closely  reviewed  and  evaluated.  It  is  by  understanding 
the  manner  in  which  the  cost  estimates  are  derived  that  a measure  of  confidence  may  be 
obtained  for  using  them. 
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X.  INTEGRATED  LOGISTICS  SUPPORT  (1LS) 

The  ILS  functions  are  essential  to  the  successful  integration  of  an  equipment 
into  operational  use.  The  ILS  concepts  must  be  initiated  in  conjunction  with  the  equip- 
ment design  concept.  All  too  often,  the  consideration  of  the  ILS  is  postponed  into  the 
development  phase  or  later;  at  this  late  stage,  the  ILS  tasks  become  little  more  than 
analysis  of  an  existing  design  to  determine  what  must  be  done  to  support  it.  The  sup- 
port system  resources  are  limited,  and  design  changes  are  expensive  to  implement.  So. 
the  most  effective  way  of  implementing  ILS  is  to  design-in  the  necessary  equipment 
features  initially.  This  requires  integrated  planning  of  design  for  performance  and  design 
for  support. 

The  ILS  disciplines  include  the  following: 

Reliability 

Maintainability 

Human  engineering 

Safety  engineering 

Transportability 

Personnel  and  training  design 

Test  equipment,  test  criteria,  facilities,  and  technical  data  planning 

Logistics  design  (including  logistics  support  analysis,  level-of-repair  analysis,  and 
standardization  engineering) 

Assistance  in  these  disciplines  is  usually  available.  NOSC,  for  example,  has  special- 
ists in  the  Product  Assurance  Division  and  the  Design  Engineering  Division.  Their  participa- 
tion should  be  included  to  an  appropriate  degree  in  any  project  developing  equipments  for 
fleet  use.  The  project  manager  should  become  familiar  with  ILS  concepts;  the  Integrated 
Logistics  Support  Implementation  Guide  for  DoD  Systems  and  Equipments  (NAVMAT 
PP4000)  is  a readily  available  reference. 


THE  ILS  CONCEPT 


CONCEPT 

The  ILS  concept  is  concerned  with  definition,  optimization,  and  integration  by 
systematic  planning,  implementation,  and  management  of  logistic  support  resources 
throughout  the  system  li'V  cycle.  The  concept  is  realized  through  the  proper  integra- 
tion of  logistic  support  elements  with  each  other  and  through  the  application  of  logis- 
tics considerations  to  the  decisions  made  on  the  design  of  the  hardware  system  and 
equipment  as  part  of  the  system  engineering  process. 

OBJECTIVE 

It  has  long  been  obvious  to  those  responsible  tor  the  operation  of  military  sys- 
tems and  equipments  that  support  problems  are  a limiting  factor  on  operational  capability 
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availability.  Much  effort  is  expended  in  trying  to  increase  mean  time  between  failures 
or  decrease  periodic  maintenance,  and  to  reduce  maintenance  downtime.  Operational 
commanders  watch  carefully  the  statistics  on  those  items  of  equipment  which  are  not 
operationally  ready  because  of  maintenance  or  supply  difficulties.  They  recognize  the 
importance  of  having  personnel  who  are  adequately  trained  to  operate  the  equipment 
properly  and  to  maintain  it  efficiently  in  order  to  reduce  the  number  and  frequency  of 
failures  and  to  reduce  the  adverse  effect  of  such  failures  and  maintenance  time  on  op- 
erational readiness.  They  know  the  importance  to  their  operation  of  adequate  facilities 
and  support  equipment. 

The  ILS  concept  must  be  applied  throughout  the  acquisition  cycle  to  ensure  that 
systems  are  designed  to  meet  operation  requirements.  Frequently  systematic  considera- 
tion of  the  solution  to  the  problems  of  support  does  not  begin  until  the  system  is  in  the 
production/deployment  phase.  While  some  elements  of  support  may  receive  early  atten- 
tion, it  is  rare  that  total  support  planning  has  a major  impact  on  system  design.  This 
lack  of  timely  and  systematic  planning  adversely  affects  operational  availability  and  cost 
of  ownership. 

Under  the  ILS  concept,  the  importance  of  trading  off  operation  and  support  re- 
quirements from  the  earliest  phases  of  the  life  of  a system  has  been  recognized.  As 
DoD  Directive  4100.35  states:  “Over  the  life  cycle  of  a system,  support  represents  a 
major  portion  of  the  total  cost,  and  is  sometimes  the  principal  cost  item.”  By  integra- 
tion of  logistics  considerations  into  the  conceptual  planning  and  through  the  entire  de- 
sign and  development  process,  either  support  costs  during  the  operation  may  be  signifi- 
cantly reduced,  or  operational  availability  of  the  system  may  be  increased  without  a 
significant  increase  in  cost. 

In  addition  to  integrating  support  planning  into  the  entire  design  and  develop- 
ment process,  it  is  also  fundamental  to  the  ILS  concept  that  the  elements  of  logistic 
support  (as  listed  and  defined  in  appendix  B)  shall  be  integrated  with  each  other  into 
a total  support  system.  When  the  baseline  of  any  one  logistics  element  is  changed  or 
proposed  to  be  changed,  the  effect  on  all  other  logistics  elements  and  on  the  total  sys- 
tem must  be  formally  considered  and  necessary  adjustments  made. 

In  applying  the  concept  of  ILS  to  a system/equipment  acquisition,  it  is  impor- 
tant to  maintain  a proper  perspective  — to  bear  in  mind  that  logistics  support  is  not  an 
end  in  itself,  but  exists  only  to  support  the  operation  of  the  system/equipment  to  which 
it  is  related.  The  support  problem  will  vary  according  to  the  complexity  and  value  of 
the  system/equipment.  Planning  for  support  must  be  tailored  to  each  acquisition  indivi- 
dually; this  guide  addresses  the  differences  of  approach  for  major  acquisition,  iess-than- 
major  acquisitions,  off-the-shelf  items,  and  modification  programs. 

It  is  also  necessary  to  bear  in  mind  that  in  any  acquisition  which  includes  devel- 
opment. there  are  two  entirely  different  types  of  effort:  first  is  the  conceptual  and 
broad  planning  stage;  second  is  the  period  from  full-scale  development  through  final  dis- 
position, in  which  the  actions  contemplated  in  the  first  stage  are  refined  and  implemented. 
Just  as  support  planning  must  be  tailored  to  the  type  of  acquisition,  it  must  also  be 
tailored  to  the  time  phasing  of  the  acquisition  process. 

The  first  part  of  the  logistics  problem  in  an  acquisition  is  to  establish  basic  char- 
acteristics which  will  enable  the  operational  requirements  to  be  achieved.  Program  man- 
agers must  keep  the  operational  mission  clearly  in  view  during  the  early  stages,  and  they 
should  recycle  and  refine  their  planning  to  determine  what  is  the  minimum  which  must 
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be  accomplished  prior  to  full-scale  development.  Once  the  basic  logistics  system  char- 
acteristics are  formulated,  they  must  be  stated  to  the  design  engineers  in  a “design  to" 
or  "design  constraint”  fashion.  When  requirements  are  stated  in  this  format,  they  may 
be  used  in  analytical  and  tradeoff  studies.  In  the  development  of  the  logistic  support 
concepts  and  the  early  planning  for  support,  program  managers  must  assure  that  logistic 
and  design  personnel  work  together  in  an  atmosphere  of  maximum  cooperation  and  com- 
munication. Thus,  the  1LS  function  must  be  closely  identified  as  an  integral  part  of  the 
total  system  engineering  process. 

The  logistics  effort  in  the  early  stages  must  be  confined  to  development  and  for- 
mulation of  inclusive  but  broad  logistics  plans  and  support  characteristics.  The  result 
should  be  a road  map  of  what  specific  steps  will  be  taken,  at  what  time,  and  in  what 
detail  as  the  development  progresses  and  the  design  matures.  The  detailed  planning  and 
preparation  of  detailed  data  packages  must  be  deferred  until  the  configuration  of  the 
hardware  has  been  reasonably  stabilized.  Detailed  support  planning  which  is  accom- 
plished prior  to  the  establishment  of  the  basic  configuration  and  dependent  on  that  con- 
figuration is  almost  certain  to  require  extensive  rework  to  become  valid  and  usable. 

The  techniques  for  the  application,  testing,  and  demonstration  of  ILS  planning 
and  the  requirements  for  the  management  of  the  logistics  effort  at  various  stages  of 
the  acquisition  process  are  covered  in  greater  detail  in  the  ILS  Implementation  Guide 
(NAVMAT  P4000)  previously  referenced. 

RELIABILITY  PROGRAM 

Reliability  is  the  probability  that  an  item  will  perform  its  intended  function  for 
a specific  interval  within  a stated  operational  environment.  To  achieve  the  required 
level  most  effectively,  the  project  manager  must  establish  a reliability  program  in  the 
early  phases  of  the  development.  He  must  ensure  that  the  designer  achieves  the  maxi- 
mum in  reliability  which  is  available  to  him;  that  the  most  reliable  components  avail- 
able are  used;  that  reliability  is  emphasized  in  all  succeeding  phases  of  development; 
and  that  design  revisions  to  incorporate  modern  technology  are  incorporated  at  the  ap- 
propriate stage  of  maturity.  He  must,  in  short,  program  reliability  from  concept  for- 
mulation to  production. 

A reliability  program  consists  of  plans  and  tasks  scheduled  in  a manner  to  pro- 
vide control  over  all  factors  affecting  the  reliability  of  equipment  during  conceptual  de- 
sign and  feasibility  demonstration,  development,  preproduction,  and  production  to  en- 
sure that  the  quantitative  reliability  requirements  of  the  equipment  are  met. 

The  project  manager  bases  his  reliability  program  on  two  basic  commandments: 

• Keep  equipment  simple  eliminate  the  superfluous. 

• Determine  minimum  acceptable  performance  levels  specify  nothing  that 
exceeds  them. 

Observance  of  these  commandments  simplifies  design,  fabrication,  operation, 
and  support;  lowers  development,  purchase,  and  support  costs;  and  maximizes  reliability. 

Detailed  reliability  program  requirements  are  provided  in  MIL-STD-7K5  and  Mil  - 
R-22732.  See  figure  X-t. 
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Figure  X-l.  Reliability  documents. 


THE  RELIABILITY  PROGRAM  PLAN 


MANAGEMENT  TASKS 

The  reliability  program  plan  should: 

• Identify  the  organizational  elements  and  the  key  personnel  responsible  for  man- 
aging the  overall  reliability  program. 

• Clearly  define  the  related  responsibilities  and  functions,  including  both  policy 
and  action. 

• Stipulate  the  authority  delegated  to  the  organizations  to  enforce  its  policies. 

• Identify  the  relationships  between  line,  service,  staff,  and  policy  organizations. 

Specific  management  tasks  called  out  in  M1L-STD-785  for  the  reliability  plan  in- 
clude: 


A detailed  listing  and  description  of  each  •'ask. 

A reference  list  of  detail  :d  procedures,  with  summary  descriptions,  to  evaluate 
the  status  and  control  of  each  task  including  identification  of  the  organizational 
unit  with  the  authority  and  responsibility  for  executing  each  task. 

The  anticipated  man-hours  required  for  each  task. 

Identification  of  known  reliability-oriented  problems  to  be  solved,  an  assessment 
of  the  impact  of  these  problems  on  specified  program  requirements,  and  the  pro- 
posed solutions  or  the  proposed  program  to  solve  these  problems. 

Procedures  for  recording  the  status  of  actions  to  resolve  the  problems. 

Designation  of  progressive  reliability  values  and  milestones,  definition  of  inter- 
relationships, and  estimation  of  times  needed  for  each  reliability  program  acti- 
vity or  task.  (Where  PERT  is  used  in  the  total  program,  appropriate  reliability 
milestones  should  be  included  in  the  overall  network.) 

Method  by  which  the  reliability  requirements  are  disseminated  to  designers  and 
associated  personnel. 

Provisions  for  reliability  indoctrination  and  training  in  connection  with  the  proj- 
ect. 


RELIABILITY  TASKS 

Reliability  tasks  germane  to  a reliability  program  are: 

Program  planning 
Design  guides 
Mathematical  modeling 
Allocation 
Prediction 

Failure  modes  and  effects  analysis 


Parts  program 
Tradeoff  studies 
Contractor  control 

Documentation  and  data  control  plans 
Design  reviews 

Developmental  test  planning  and  testing 
Failure  analysis  during  testing 

Most  of  these  tasks  are  performed  during  the  concept  design  and  developmental 
stages  of  a program  before  a preproduction  design  is  formulated.  Application  of  the 
tasks  during  each  acquisition  phase  is  described  in  MIL-R-22732.  A number  of  these 
tasks  are  further  described  below. 


ALLOCATION.  Allocation  is  apportionment  of  equipment  reliability  from  an 
individual  equipment  specification  to  lower  limits  within  the  equipment.  This  is  accotr. 
plished  by  allocating  numerical  reliability  goals  to  each  assembly  and  subassembly  down 
to  each  nonrepairable  part.  When  combined  with  the  mathematical  model,  the  allocated 
goals  should  yield  an  equipment  reliability  not  less  than  that  required  in  the  individual 
equipment  specification.  A reliability  group  usually  performs  this  task  and  the  results 
provide  the  project  manager  with  his  numerical  reliability  requirement. 


PRFDICTION  Prediction  is  the  determination  of  equipment  reliability  from  the 
reliability  characteristics  of  its  components.  Reliability  predictions,  performed  early  in 
the  design  phase,  are  used  as  a basis  for  determining  the  adequacy  of  the  equipment  con- 
figurations and  models  for  meeting  the  allocated  design  goals.  Prediction  makes  it  pos- 
sible to  determine  the  weak  links  in  an  equipment’s  reliability  and  to  determine  the  nec- 
essary changes,  their  costs,  and  the  reliability  improvement  to  be  expected  from  their  ac- 
complishment. Subtasks  of  prediction  are:  (1)  reliability  logic  block  diagram  (which 
shows  the  relationships  between  equipment  operation/failure  and  constituent  equipment 
or  component  operation/failure)  and  (2)  mathematical  model  of  the  logic  block  diagram 
in  equation  form.  MIL-STD-756  shows  how  to  construct  the  logic  block  diagram  and 
make  predictions,  while  MIL-HDBK-217  provides  amplification,  elaboration,  and  a data 
base.  See  figure  X-l. 


FAILURF.  MODFS  AND  EFFECTS  ANALYSIS  (FMF.A).  FMHA  determines  the 
effects  on  hardware  (circuit)  outputs  when  constituent  parts  fail  in  different  modes.  Typ- 
ical part  failure  modes  are  fail  open,  fail  closed,  and  parameter  drift.  Examples  are  given 
in  NAVSHIPS  0967-316-8010  See  figure  X-l.  This  task  points  out  to  the  project  man- 
ager the  effects  each  item  failure  has  on  the  design  and  the  manner  in  which  the  item  tails. 
Failures  which  might  appear  to  be  simple  could  be  critical  to  system  operation.  This  task 
allows  the  designer  to  soften  or  remove  the  impact  of  the  failure  on  his  design  if  the  item 
is  determined  to  be  critical  to  system  operation.  The  FMF.A  results  combined  with  the 
prediction  task  will  provide  information  to  the  project  manager  on  the  reliability  worthi- 
ness of  the  design  and  the  weak  links  in  the  reliability  chain. 
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From  this  information  he  can  make  more  reasonable  decisions  as  to  the  need 
for  design  changes  and.  if  needed,  the  types  or  areas  of  change  that  will  yield  a signi- 
ficant improvement  in  reliability.  F'MFA  is  used  as  a tool  tor  design  improvements  to 
eliminate  or  reduce  item  criticality  by  providing  redundancy,  alternate  modes  of  oper- 
ation. or  increased  personnel  and  material  safety. 

TEST  PLANNING  AND  TESTING.  The  proof  of  achieved  reliability  and.  con- 
versely, the  uncovering  of  deficient  areas  of  design,  lie  in  the  testing  of  the  item.  The 
designer  should  gather  appropriate  data  for  reliability  purposes  during  the  development 
and  testing  stages  Measures  such  as  accept/reject  criteria,  the  definition  of  a failure,  and 
instrumentation  and  data  requirements  should  be  established. 

The  designer  should  also  supply  information  which  indicates  the  types  of  tests 
to  speeifv . test  equipment  required  for  the  test,  acceptable  limits  of  operation,  and  type 
of  test  report  required.  Testing  should  be  performed  at  the  environmental  stresses  listed 
m the  individual  equipment  specification.  These  inputs  allow  the  test  and  evaluation 
group  to  develop  a reliability  test  plan  as  described  in  MIL-STD-781.  See  figure  X- 1 


I \ 1 1 URL  ANALYSIS  DURING  II  STING.  This  task  determines  the  following 
t i ) estimates  ot  the  reliability  of  hardware  from  test  data.  (2)  whether  the  equipment 
is  to  be  accepted  or  rejected,  and  (3)  causes  of  failure  and  weaknesses  in  design.  (See 
Mil  SID-^Sl  i It  provides  the  basic  data  for  the  design  analysis  and  redesign  of  equip- 
ment It  provides  the  feedback  loop  to  project  managers  so  they  can  effect  a design 
change  that  eliminates  the  uncovered  deficiencies. 

1)1  SIGN  Rl  VII  WS.  Formal  and  informal  design  reviews  provide  the  necessary 
interaction  between  designers,  project  managers,  sponsors,  and  users  that  permits  an  in- 
sight into  the  designer’s  ideas  and  allows  an  appraisal  of  his  approach,  progress,  and 
problems  Design  reviews  provide  the  designer  a more  precise  understanding  of  the  user's 
requirements  and  problems  and  of  whether  or  not  his  design  approach  will  fulfill  the  re- 
liability needs  of  the  user.  Formal  design  reviews  usually  consist  of  a Preliminary  De- 
sign Review,  held  during  the  preliminary  design  of  the  equipment,  the  Critical  Design 
Review,  usually  held  70  days  prior  to  formal  design  release,  and  the  Final  Design  Review  . 
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MAINTAINABILITY  PROGRAM 

GENERAL 

Maintainability  is  that  part  of  equipment  design  which  contributes  to  the  rapidity, 
ease,  and  economy  of  maintenance  and  repair.  It  provides  design  features  and  functions 
for  simplifying  or  expediting  the  maintenance  tasks  which  must  be  performed  in  order 
to  keep  an  equipment  in  its  specified  operating  condition  or  to  restore  it  to  that  condi- 
tion in  the  operating  environment.  A high  level  of  maintainability  not  only  reduces 
equipment  downtime  but  also  helps  reduce  the  life-cycle  cost  of  maintenance  and  logistic 
support. 

Early  in  equipment  design,  the  project  manager  arranges  for  a maintenance  engi- 
neering analysis  to  be  performed  in  accordance  with  MIL-M-24365.  See  figure  X-2 

The  project  manager  makes  certain  that  the  designer  is  aware  of  both  the  esta- 
blished shipboard  maintenance  procedures  and  the  physical  conditions  under  which  main- 
tenance is  to  be  performed;  and  that  he  is  aware  of  the  qualifications  and  limitations  ol 
the  technicians  who  will  maintaivi  the  equipment. 

Early  in  equipment  design,  the  project  manager  arranges  for  a maintainability  pro- 
gram to  be  set  up  in  accordance  with  MIL-STD-470. 

The  maintainability  program  provides  that  the  contractor  must: 

Prepare  a maintainability  program  plan. 

Perform  a maintainability  analysis  (distinct  from  the  maintenance  engineering 
analysis).  A major  task  of  the  maintainability  analysis  is  the  allocation  ol  quan- 
titative maintainability  requirements  to  all  significant  levels  of  the  system. 

Prepare  inputs  lo  the  detailed  maintenance  concept  and  detailed  maintenance 
plan. 

Establish  maintainability  design  criteria. 

Perform  design  tradeoffs. 


Figure  X-2.  Maintainability  documents. 

Predict  maintainability  parameter  values.  Techniques  lor  prediction  are  found  in 
M1L-HDBK-472. 

Incorporate  and  enforce  maintainability  requirements  in  subcontractor  and  vendor 
specifications. 

Integrate  items  other  than  the  contractor’s  items  into  the  maintainability  program 

Participate  in  design  reviews  held  at  appropriate  stages  of  the  equipment  develop- 
ment. 

Establish  a data  collection,  analysis,  and  corrective  action  system. 

Demonstrate  the  achievement  of  maintainability  requirements.  MIL-STD-471  gives 
the  procedures,  test  methods,  and  requirements  for  verification,  demonstration, 
and  evaluation  of  the  achievement  of  the  specified  maintainability  requirement 

Prepare  maintainability  status  reports  at  intervals  determined  or  approved  by  the 
procuring  activity. 

MAINTAINABILITY  REFERENCES 

Naval  Air  Systems  Command  Military  Standard  Mil -STD-470.  Maintainability 
Program  Requirements  (Lor  Systems  and  I qntpmcnls).  ’I  Mar  UP 

Naval  Air  Systems  Command  Military  Standard  Mil  -STD-471A.  Maintainability 
Demonstration.  2 7 Mar  73 
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Naval  Air  Systems  Command  Military  Handbook.  MIL-HDBKjJ72.  Maintainability 
Prediction.  24  May  66 

Naval  Ship  Engineering  Center  Military  Specification  M/L-M-24365A.  Maintenance 


Naval  Ship  Engineering  Center  Military  Standard  MIL-STD-881  A.  Work  Breakdown 
Structures  for  Defense  Materiel  Items.  25  Apr  75 


Naval  Sea  Systems  Command  Military  Specification.  MIL-P-24534,  Planned  Maintenance 
Subsystem;  Development  of  Maintenance  Requirement  Cards.  Maintenance  Index  Pages,  and 
Associated  Documentation,  26  April  1976 


HUMAN  ENGINEERING  PROGRAM 

GENERAL 

Electronic  equipment  should  be  designed  to  permit  full  utilization  of  human  capa- 
bilities. to  compensate  for  human  limitations,  and  to  eliminate  the  possibility  of  human 
error. 

The  project  manager  arranges  for  comments  to  be  solicited  from  operating  and 
maintenance  personnel  during  the  early  planning  stages. 

The  project  manager  arranges  for  operator  interaction  with  equipment  during  ad- 
vanced and  engineering  development  to  aid  in  finding  and  eliminating  human  en- 
gineering problems  before  the  service  test  demonstration  and  final  production  de- 
sign specifications. 

The  project  manager  administers  the  human  engineering  program.  He  is  conver- 
sant with  the  provisions  of  MIL-H-46855,  which  presents  human  engineering  re- 
quirements including  analysis,  test  and  evaluation,  program  plan,  physical  models, 
procedure  development,  and  coordination  with  other  programs. 

The  project  manager  is  also  conversant  with  the  provisions  of  MIL-STD-1472. 
which  presents  human  engineering  design  criteria.  See  figure  X-3. 


MIL-H-46855 


MIL-STD  1472 
DESIGN  CRITERIA 


Figure  X-3  Human  engineering  documents. 
X-10 
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HUMAN  ENGINEERING  REFERENCES 


Naval  Air  Systems  Command  Military  Specification  MIL-H-46855A,  Human  En- 
gineering Requirements  for  Military  Systems,  Equipments  and  Facilities. 

2 May  72 

Naval  Air  Systems  Command  Military  Standard  MIL-STD-I472B.  Human  Engi- 
neering Design  Criteria  for  Military  Systems,  Equipment  and  Facilities, 

I December  74 

US  Army  Military  Handbook  M1L-HDBK-759,  Human  Factors  Engineering  Design 
for  A rim  Material.  12  March  1975 


SAFETY  PROGRAM 


GENERAL 

The  chief  objectives  of  the  safety  program  are  the  prevention  of  injury  to  per- 
sonnel and  the  prevention  of  damage  to  equipment. 

The  project  manager  determines  whether  or  not  to  specify  that  the  equipment 
contractor  develop  and  maintain  an  effective  system  safety  program  as  outlined 
in  MIL-E-16400.  Preparation  of  the  program  is  covered  in  MIL-STD-882.  The 
primary  reference  for  safety  criteria  is  requirement  1 of  MIL-STD-454.  See 
figure  X-4. 

The  project  manager  is  willing  to  expose  himself  to  the  worst-case  personnel 
hazards  the  equipment  he  is  responsible  for  is  capable  of  presenting  to  Navy 
personnel. 

Whether  or  not  a formal  system  safety  program  is  established  for  the  project,  safety  should 
be  an  issue  integral  to  all  design  reviews.  The  equipment  design  and  operational  procedures 
should  be  scrutinized  for  conformance  to  safety  criteria.  The  actions  considered  should 
include,  but  not  be  limited  to.  the  following: 

1.  Avoiding,  eliminating,  or  reducing  identified  hazards  by  analysis,  design  selec- 
tion. material  selection,  or  substitution 

2.  Controlling  and  minimizing  hazards  to  personnel,  equipment,  and  material 
which  cannot  be  avoided  or  eliminated 

3.  Isolating  hazardous  substances,  parts,  and  operations  from  other  activities, 
areas,  personnel,  and  incompatible  materials 

4.  Incorporating  “failsafe”  principles  where  failures  would  disable  the  system  or 
cause  a catastrophe  through  injury  to  personnel  or  damage  to  equipment 

5.  Locating  equipment  parts  so  that  access  to  them  by  personnel  during  opera- 
tion, maintenance,  repair,  or  adjustment  shall  not  require  exposure  to  hazards 
such  as  chemical  burns,  electrical  shock,  cutting  edges,  sharp  points,  or  toxic 
atmospheres 

().  Avoiding  undue  exposure  of  personnel  to  physiological  and  psychological 
stresses  which  might  cause  errors  leading  to  mishaps 
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7.  Providing  warning  and  caution  notes  in  operations,  assembly,  maintenance,  and 
repair  instructions;  and  distinctive  markings  on  hazardous  parts  equipment,  or 
facilities  for  personnel  protection 

8.  Minimizing  damage  or  injury  to  personnel  and  equipment  in  the  event  ol  an 
accident 

In  satisfying  safety  requirements,  the  design  solutions  should  follow  this  order  ol 
precedence: 

1 . Design  for  minimum  hazard 

2.  Safety  devices 

3.  Warning  devices 
Special  procedures 

MIL-1  -lt>400  specifies  the  following  tasks  (they  are  worthy  of  consideration  even  when 
MIL-E-1P400  does  not  apply): 

I.  Safety  testing 

2 Design  review  using  a system  safety  checklist 

3.  Safety  analyses 

a.  Conceptual  safety  analysis  (CSA) 
h.  Design  safety  analysis  (DSA) 

e.  Functional  safety  analyses  considering,  as  a minimum. 

Installation  requirements 

Testing  requirements 

Operating  and  maintenance  requirements 

Safety  supervision 

Handling  requirements 

Tra i n i ng  req uiremen t s 

d.  Hazard  Mode  and  Effects  Analysis  (HMEA) 

4.  Integration  with  other  project  tasks 


MIL  E 16400 
3.10 

PROGRAM 


MIL-STD-454 
REQ  1 
CRITERIA 


MIL-STD-882 

PROGRAM  REQUIREMENTS 


Figure  X-4.  Safety  documents. 
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SAFETY  REFERENCES 


Naval  Ship  Engineering  Center  Military  Specification  MIL-E-16400G  (NAVY), 
Electronic.  Interior  Communication  and  Navigation  Equipment,  Naval  Ship  and 
Shore.  General  Specification  for,  24  Dec  74 

Naval  Air  Systems  Command  Military  Standard  MIL-STD-454I),  Standard  Gen- 
eral Requirements  for  Electronic  Equipment,  31  Aug  73 

Naval  Air  Systems  Command  Military  Standard  MIL-STD-882,  System  Safety 
Program  for  Systems  and  Associated  Subsystems  and  Equipment:  Requirements 
for.  15  Jul  69 


PACKAGING,  HANDLING,  STORAGE,  AND 
TRANSPORTABILITY  (PIIST)  PROGRAM 

GENERAL 

The  PHST  program  considers  all  the  problems  of  transporting  the  system  or  equip- 
ment from  the  development  site  to  the  test  site,  from  the  production  line  to  storage,  and 
from  storage  to  service  use.  MIL- STD-1 367  contains  the  PHST  program  requirements.  The 
program  can  be  viewed  as  being  in  two  phases  — analysis  and  design.  The  analysis  phase 
determines  what  transportation,  handling,  and  storage  actions  will  be  used  through  the  life 
of  the  equipment.  The  design  phase  determines  the  detailed  PHST  requirements  and  devel- 
ops testing,  packaging,  and  handling  procedures  for  use  with  the  equipment  as  well  as  incor- 
porating design  features  into  the  equipment  to  facilitate  the  procedures.  Figure  X-5  shows 
the  relationships  of  the  primary  reference  documents. 

MIL  -STD  1319 


CONSIDERATIONS  AIR  TRANSPORTABILITY 

REQUIREMENTS 

Figure  X-5.  PMST  documents. 

REFERENCES 

Dol)  Military  Sid,  M1L-STD-1 367,  Packaging,  Handling.  Storage.  & Transportability 
Program  Requirements.  27  April  1677 

Dol)  Military  Std.  MIL- STD- 1 366.  Packaging.  Handling.  Storage,  & Transportation 
System  Dimensional  Constraints.  27  April  I 672 
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DoD  Military  Std,  MIL-STD-1365,  General  Design  Criteria  for  Handling  Equipment 
Associated  with  Weapons  and  Weapon  Systems,  27  April  1972 

DoD  Military  Std,  M1L-STD-794D.  Procedures  for  Packaging  and  Packing  of  Parts 
and  Equipment,  15  December  1972 

DoD  Military  Std,  M1L-STD-810C,  Environmental  Test  Methods,  10  March  1975 

US  Air  Force  Military  Std,  M1L-P-9024G.  Packaging,  Handling,  and  Transportability 
in  System/Equipment  Acquisition,  6 June  1972 

US  Air  Force  Military  Std,  M1L-A-8421F.  Air  Transportability  Requirements, 

25  October  1974 

Naval  Air  Engineering  Center  Military  Spec,  MIL-P-1  16G.  Methods  of  Preservation- 
Packaging,  27  June  1975 

DoD  Military  Std,  M1L-STD-13 19A,  Item  Characteristics  Affecting  Transportability 
and  Packaging  and  Handling  Equipment  Design,  21  August  1974 

Naval  Ship  Engineering  Center  Military  Spec,  MIL-E-1 7555G,  Packaging  and  Packing 
of  Electronic  and  Electrical  Equipment,  Accessories,  and  Repair  Parts,  3 July  1968 

US  Navy  Military  Std.  M1L-STD-1 342,  Preparation  for  Delivery  of  Limited  Shelf 
Life  Electronic  Equipment  and  Parts,  15  August  1969 


PERSONNEL  AND  TRAINING  PROGRAM 


GENERAL 

The  personnel  and  training  program  implements  the  “human  equation”  of  the  opera- 
tion and  maintenance  concepts.  The  program  starts  with  an  analysis  of  O&M  concepts  to 
determine  the  skills  and  skill  levels  needed  to  support  the  system.  Reliability  program  in- 
puts help  to  establish  the  man-hour  demands  for  each  skill.  The  man-hour  demands  are  used 
to  determine  whether  additional  personnel  will  be  required  in  the  user  organization  and  at 
support  facilities.  The  user  and  support  organizations  are  then  compared  to  the  projected 
skill  requirements  to  determine  tentative  training  requirements. 

A training  program  plan  is  then  prepared  in  accordance  with  OPNAVINST  1500.8 
series.  OPNAVINST  I 500.8  and  OPNAVINST  1 500.44  together  establish  the  procedures 
for  coordinating  project  requirements  with  the  training  commands.  It  takes  1 year  or  more 
to  establish  the  final  training  requirements  and  2 years  to  budget  and  to  start  training  imple- 
mentation. The  training  plan  also  addresses  interim  training  measures  for  teams  to  support 
test  and  evaluation  and  for  instructors  and  the  initial  students  to  cover  the  gap  between  the 
equipment  IOC  and  the  establishment  of  the  final  training  support. 

OPNAVINST  1500.2  dictates  the  policies  and  procedures  for  the  “factory”  training 
programs  which  are  normally  needed  to  meet  interim  training  requirements.  Detailed  train- 
ing plans,  course  plans,  training  aids,  and  course  materials  are  prepared  in  accordance  with 
Ml  L -STD-1 379.  Interim  training  is  an  essential  project  task  which  must  be  fully  integrated 
into  the  overall  project  plans,  budgets,  and  schedules.  This  training  provides  key  support  to 
final  operational  testing  and  for  phasing  the  equipment  into  service  use. 
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US  Navy  Military  Std,  M1L-STD-1 379,  Training  Operations  and  Training  Data, 

15  March  1972 

OPNAVINST  I 500.SU  of  3 July  1975,  Preparation  and  Implementation  of  Navy 
Training  Plans  (NTPs)  in  Support  of  Hardware  and  Non-hardware  Oriented 
Developments 

Bureau  of  Naval  Personnel  NAVPERS  18068D.  Manual  of  Qualifications  for  Ad- 
vancement. January  1977 

Naval  Education  and  Training  Command  NAVEDTRA  10,500,  Catalog  of  Navy 
Training  Courses  (updated  frequently) 

Bureau  of  Naval  Personnel  NAVPERS  16103C,  Manual  for  Navy  Instructors.  1964 

TEST  & MEASUREMENT  PROGRAM 

GENERAL 

The  T&M  program  encompasses  a broad  base  of  tasks  required  to  support  the  main- 
tainability program.  These  tasks  include  conducting  tradeoff  studies  between  test  and  meas- 
urement methods,  selecting  test  equipments,  coordinating  the  equipment  design  to  incorpo- 
rate test  features  and  test  points  adequate  for  maintenance,  and  generating  and  documenting 
test  procedures  for  organizational  and  depot  use.  The  T&M  program  and  the  maintainabili- 
ty program  are  major  determinants  of  the  maintenance  design  and  ol  the  effectiveness  of  the 
maintenance  concept  implementation. 

The  primary  standards  used  by  the  T&M  program  are  MIL-STD-1345  and  MIL-SI  D- 
41 5.  Tradeoffs  are  established  between  different  test  methods  using  maintenance  time, 
calibration  requirements,  skill  requirements,  equipment  reliability,  and  cost  criteria.  The 

final  design  may  incorporate  any  or  all  of  the  following  general  methods:  | 

Automatic  Test  Equipment  (ATE)  1 

Built-In  Test  Equipment  (BITE) 

Test  Points  with  General-Purpose  Electronic  Test  Equipment  (GPETE) 

Test  Points  with  Special-Purpose  Electronic  Test  Equipment  (SPETE) 

The  use  of  general-purpose  ATE  systems  such  as  the  Versatile  Avionics  Shop  Tester  (VAST) 
or  the  Integral  Sensor  Test  System  ( I STS » may  be  dictated  by  platform  maintenance  re- 
quirements. The  use  of  software  diagnostics  within  the  system  may  be  very  desirable. 

There  is  no  best  solution  for  all  cases,  so  these  project  tasks  are  important.  In  each  case, 
special  design  provisions  will  be  dictated  by  the  test  methods  employed.  The  impacts  on 
training,  technical  manual,  logistics,  and  facility  requirements  need  to  be  considered  and 
coordinated  with  those  ITS  tasks.  Unless  justified  by  other  characteristics,  the  T&M  pro- 
gram will  endeavor  to  eliminate  special  tools,  test  equipment,  and  facilities. 
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REFERENCES 

DoD  Military  Std.  MIL- STD-41 5D,  Design  Criteria  for  Test  Provisions  for  Electronic 
Systems  and  Associated  Equipment,  I October  1969 

Naval  Electronic  Systems  Command  Military  Std.  M1L-STD-1 345  A.  Measurement 
Data  in  Support  of  Maintenance,  Calibration,  and  Repair  of  Electronic  Equipment, 

21  March  1975 

Naval  Ship  Engineering  Center  Military  Std.  M1L-STD-1 326.  Test  Points,  Test  Point 
Selection,  and  Interface  Requirements  for  Equipments  Monitored  by  Shipboard  On- 
Line  Automatic  Test  Equipment,  15  January  1968 

US  Air  Force  Military  Std.  M1L-STD-1 536.  Integral  Sensor  Test  System.  15  Febru- 
ary 1973 

US  Navy  Military  Std.  M1L-STD-1 364C,  Standard  General  Purpose  Electronic  Test 
Equipment,  30  May  1975 

LOGISTICS  PROGRAM 

GENERAL 

The  logistics  program  consists  of  three  subprograms  the  support  analysis  program, 
the  provisioning  program,  and  the  standardization  program.  The  purpose  ol  the  logistics 
program  is  twofold : 

• To  drive  the  equipment  design  toward  the  most  economic  methods  ot  support 

• To  plan  and  to  implement  the  support  of  each  equipment  component 

The  man\  tasks  of  the  logistics  program  are  so  intertwined  with  the  other  1LS  programs  that 
it  is  normally  managed  directly  by  the  project  1LS  manager.  Like  the  other  1LS  specialties, 
the  logistics  program  can  be  very  complex  or  relatively  simple,  and  it  should  be  tailored  to 
the  project  requirements.  The  extent  of  the  support  analysis  program  is  especially  suscepti- 
ble to  tailoring  techniques.  However,  the  goals  of  the  logistic  program  must  be  satisfied  in 
order  to  transition  the  equipment  into  service  use. 

The  standardization  tasks  are  so  intimately  connected  with  the  management  ot  the 
design  effort  that  they  are  provided  separate  treatment  in  chapter  XIV.  Consideration  ot 
Standardization.”  Also,  parts  selection  and  management  concepts  are  discussed  in  chapter 
XXII.  Refer  to  tiiese  sections  for  a detailed  discussion  of  standardization  program  tasks  and 
issues. 

SUPPORT  ANALYSIS  PROGRAM 


The  support  analysis  program  verifies  the  general  support  concepts  and  synthesizes 
detailed  support  plans  by  analyzing  the  requirements  imposed  by  operational,  design,  and 
the  other  II  S programs.  Support  effectiveness  and  costs  are  the  main  decision  (actors; 
life-cycle  costs  comprise  90' • of  the  cost  factors.  In  the  conduct  of  the  analysis  program, 
alternative  methods  of  support  are  investigated,  and  the  implications  on  designs  are  led  back 
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to  the  system  engineer.  System  engineering  conducts  the  necessary  trade  studies  and  either 
imposes  design  changes  or  design  freezes  sufficient  to  establish  the  support  concept  baseline. 

There  are  two  primary  analysis  techniques  employed  in  the  support  analysis  pro- 
gram: the  logistic  support  analysis  ( LSA)  and  the  level  of  repair  (LOR)  analysis.  I lie  1 SA 
is  a readily  tailored  management  technique  for  coordinating  ILS  functions  as  well  as  accom- 
plishing the  support  analysis  itself.  MIL-STD-1369  contains  LSA  requirements,  hut  a more 
detailed  breakdown  of  tasks  and  procedures  is  documented  in  M1L-STD-13S8.  1 lie  I OK 
analysis  is  a life-cycle-cost  based  technique  for  establishing  the  best  location  for  accomplish- 
ing repair  tasks.  MIL-STD-1390  contains  the  detailed  formulas  used  for  LOR  analysis. 

Many  of  the  LOR  inputs  come  from  lSA  computations,  and  the  LOR  output  is  used  in  later 
LSA  iterations. 


PROVISIONING  PROGRAM 

The  provisioning  program  provides  for  the  detailed  analysis  of  each  repair  part  sup- 
port requirement  and  documents  the  results  in  the  appropriate  lists  and  codes  for  use  by  the 
supply  system.  The  support  analysis  program  will  determine  the  general  sparing  philosophy, 
but  the  provisioning  program  fills  out  the  details.  The  various  code  assigned  to  each  part 
will  determine  the  cost  and  supply  effectiveness  of  supporting  that  part.  The  provisioning 
program  attempts  to  maximize  the  support  effectiveness  for  the  equipment  while  minimiz- 
ing the  costs  of  inventories.  These  goals  are  greatly  facilitated  by  an  effective  standardiza- 
tion program  and  by  designs  which  incorporate  built-in  spares.  The  provisioning  program 
also  prepares  plans  for  the  provisioning  conference  (sec  chapter  V 1 1 1 > and  plans  lor  interim 
support  until  the  normal  support  system  is  established  I he  interim  support  plans  may 
include  warranty  provisions  and  contractor  support  and  will  describe  the  controls  and  proce- 
dures for  monitoring  the  support. 

In  the  past,  each  service  has  had  its  own  unique  procedures  for  provisioning.  Howev- 
er. many  standard  parts  are  now  managed  by  the  Defense  Supply  \gcncy  or  are  supported 
by  a lead  service.  Therefore,  uniform  procedures  have  been  imposed  for  all  new  equipments 
and  systems  through  Ml  L- STD-150 1 . MIL-STD-1552  establishes  the  lormats  and  prepara- 
tion instructions  for  provisioning  technical  documentation.  MIL-SI  I)-I5  I 7 prescribes  the 
procedures  and  conditions  for  the  phased  provisioning  ol  interim  and  initial  spares  and 
repair  parts.  The  coding  of  repair  parts  is  specified  by  Mil  -SI  l)-7X9. 

The  most  challenging  tasks  of  the  provisioning  program  involve  investigating  and 
recommending  special  support  actions.  I hose  include  spare  support  during  development 
and  methods  of  supply  for  parts-peculiar.  critical  technology  parts,  and  long-lead-time  items 
Spare  support  during  development  is  normally  very  low  risk  and  is  usually  accomplished  by 
purchasing  some  comfortable  number  ol  spares  over  known  requirements.  Spot  ial  methods 
of  supply  may  require  purchasing  all  the  lile-cy  cle  requirements  at  once  and  setti  a up  a 
central  stock  point  ( life-of-type  (1  Of)  stocking)  or  designating  a stippoi  t man  am  i to  ston 
manufacture,  or  assemble  repair  items  from  standard  or  commercially  av  ailablc  parts 

REFERENCES 

Naval  I lev  Ironic  Systems  ( ommand  Military  Standard  Mil  S I I)- 1 do'i.  I nt  cei  a led 

Logistic  Support  Requirements.  31  March  1971 
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Department  ot  Detense  Military  Standard  MIL-STD-1388,  Logistic  Support  Analy- 
f sis,  1 5 October  1973 

Naval  Weapons  Engineering  Support  Activity  Military  Standard  MIL-STD-1 390A. 
Level  of  Repair,  1 April  1974 

Department  ot  Detense  Military  Standard  MIL-STD- 1 56 1 , Uniform  DoD  Prov  ision- 
ing Procedures,  1 1 November  1974 

Department  of  Defense  Military  Standard  MIL-STD-1 552.  Uniform  DoD  Require- 
ments for  Provisioning  Technical  Documentation.  I 1 November  1974 

Department  of  Defense  Military  Standard  MIL-STD-1 5 17.  Phased  Provisioning. 

1 June  1971 

Department  of  Defense  Military  Standard  MIL-STD-789B.  Procurement  Method 
Coding  of  Replenishment  Spare  Parts,  15  May  1970 
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XL  SCREENING  TECHNIQUE; 

INTRODUCTION  . . page  XI- 1 
CONSTRUCTING  THE  SCREEN  . . . Xl-l 
LOCATING  CANDIDATE  EQUIPMENTS  . . . XI-2 
ELIMINATING  UNLIKELY  CANDIDATES  . . . XI-3 
CONDUCTING  VISUAL  INSPECTIONS  . . . XI-3 
REVIEWING  DATA  . . . XI-3 

CONTACTING  MANUFACTURERS.  DEALERS,  AND  USERS  . . . XJ-4 
OBTAINING  UNITS  FOR  TEST  . . . XI-5 

CONDUCTING  PERFORMANCE  AND  ENVIRONMENTAL  TESTS  . . . Xl-5 
EVALUATING  THE  RESULTS  . . . XI-8 
PREDICTING  PERFORMANCE  . . . XI-8 


XI  SCREENING  TECHNIQUE.'. 


XI.  SCREENING  TECHNIQUES 


INTRODUCTION 

The  development  process  is  an  expensive  and  time-consuming  way  to  acquire  equip- 
ment; wherever  an  existing  equipment  can  be  utilized  off-the-shelf  or  after  incorporating  a 
simple  modification,  it  will  likely  prove  to  be  a more  cost-effective  alternative  than  any 
development  alternative.  However,  there  are  some  important  tradeoffs  which  must  be 
evaluated  to  establish  this  fact  for  each  specific  acquisition.  These  tradeoffs  imply  ques- 
tions which  include  the  following. 

1.  What  additional  functions,  performance  levels,  or  features  are  valuable  to  in- 
corporate over  the  minimum  requirements?  (It  is  likely  that  existing  equip- 
ments will  lack  those  features  which  might  extend  the  practical  service  life  of 
the  equipment  significantly  or  make  operation  and  maintenance  simpler.) 

2.  Will  existing  equipments  be  supportable  in  the  field? 

3.  Will  existing  equipments  be  available  in  the  future  to  support  future  acquisi- 
tion needs? 

The  effective  screen  obtains  the  information  necessary  to  answer  these  questions 
and  to  establish  the  tradeoff  decision  points  between  development  and  off-the-shelf  alter- 
natives; even  if  development  is  indicated,  a great  deal  of  valuable  information  can  be  gained 
through  the  screen  to  help  avoid  subtle  pitfalls. 

A screen  consists  of  the  following  elements: 

• Locating  candidate  equipments 

• Eliminating  unlikely  candidates 

• Conducting  visual  inspections 

• Reviewing  technical  manuals,  operating  instructions,  and  other  data  associated 
with  the  equipment 

• Contacting  manufacturers,  dealers,  and  users 

• Conducting  performance  and  environmental  tests 

The  screen  depends  heavily  on  adequately  stated  technical  requirements  for  its 
success.  Usually  the  screen  will  not  produce  a black-and-white  decision,  so  an  economic 
analysis  will  be  necessary  to  reach  a final  conclusion. 


CONSTRUCTING  THE  SCREEN 

fhe  screen  is  derived  directly  from  acquisition  requirements  consisting  of; 

Technical  requirements  contained  in  the  system  specification 

Procurement  requirements  (the  number  of  units  needed  in  the  near  term  and  in 
the  future) 

Peculiar  features  of  the  intended  application  (see  the  five  major  issues  discussed  in 
chapter  VI.  Transition  to  Production) 


Support  requirements 
Cost  constraints 

The  technical  requirements  must  fully  embrace  the  interfaces,  human  engineering 
criteria,  reliability,  maintainability,  safety,  transportability',  environmental  conditions,  and 
so  forth  which  are  dictated  by  the  operational  requirements.  Normally,  the  system  specifi- 
cation is  not  adequately  formulated  until  near  the  end  of  the  conceptual  phase  and  not 
validated  until  the  validation  phase  is  nearly  complete.  The  screen  can  be  applied  in  the 
conceptual  phase,  but  not  without  some  risk  resulting  from  inadequate  technical  require- 
ments. The  end  of  the  validation  phase  is  the  recommended  time  to  implement  the  screen, 
because  the  technical  requirements  must  be  grouped  into  "hard”  requirements  and  "soft” 
requirements.  Hard  requirements  are  all  the  characteristics,  functions,  and  parameters 
whieh  are  essential  in  the  equipment;  hard  parameters  must  be  toleranced.  Soft  require- 
ments consist  of  the  desirable  characteristics  and  functions  and  of  the  parameters  which 
may  be  "sloppy.”  It  may  be  difficult  to  group  requirements  prior  to  the  validation  phase. 
The  acquisition  requirements  can  now  be  applied  to  the  screening  steps  in  the  manner 
described  in  succeeding  sections. 
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LOCATING  CANDIDATE  EQUIPMENTS 

Candidate  military  equipments  can  be  obtained  by  determining  the  proper  nomen- 
clature(s)  for  the  desired  function  using  M1L-HDBK-140  and  then  searching  these  nomen- 
clatures in  the  Joint  Electronics  Type  Designation  File  (available  on  microfilm  in  the 
Library).  The  alphabetical  listing  of  military  specifications  may  also  prove  useful.  Some 
candidates  may  be  rejected  and  new  ones  found  through  discussions  with  cognizant  per- 
sonnel in  the  appropriate  Naval  Systems  Command,  the  Army  Electronics  Command,  or 
the  Air  Force  Electronics  Command.  The  GSA  catalog  also  lists  possible  candidates. 

Candidate  commercial  equipments  can  be  isolated  through  the  Design  Engineering 
File  of  the  Visual  Search  Microfilm  File  (VSMF)  service,  which  is  a compendium  of  supplier 
catalogs.  Other  candidates  may  be  supplied  by  the  appropriate  technical  or  industrial 
society  and  by  known  commercial  users  of  like  equipment.  The  Conover-Mast  Purchasing 
Directory  (a  Cahners  publication).  Defense  Marketing  Services,  the  Directory  of  Engineer- 
ing Document  Sources  (Global  Engineering  Document  Services  publication).  Electronic 
Industry  Telephone  Directory  (Harris  Publishing  Company,  Cleveland.  Ohio),  and  the 
Thomas  Register  are  also  useful.  For  items  over  S10  000.  a “sources  sought”  can  he 
issued  in  the  Commerce  Business  Daily;  the  CBD  advertisement  should  specifically  restrict 
respondees  to  existing  products  when  it  is  used  for  this  purpose. 

In  either  case,  the  “hard”  technical  requirements  are  the  governing  criteria  for  con- 
ducting the  search.  When  sufficient  information  is  not  available,  the  manufacturer  should 
be  queried  directly;  however,  it  is  necessary  to  take  care  not  to  disclose  information  which 
would  give  the  manufacturer  an  unfair  advantage  should  a development  be  necessary  (to 
do  so  is  a violation  of  the  Armed  Forces  Procurement  Regulations). 
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ELIMINATING  UNLIKELY  CANDIDATES 


The  following  candidates  should  lx-  eliminated: 


1.  Any  candidate  not  meeting  the  hard  technical  requirements  which  cannot 
easily  be  modified  to  do  so. 

2.  Any  candidate  which  cannot  be  supported  adequately  in  the  field  (this 
normally  applies  only  in  situations  where  depot-level  maintenance  cannot  be 
employed). 

3.  Any  candidate  which  is  not  truly  off-the-shelf.  Some  military  equipments 
are  available  only  after  a sufficient  quantity  is  needed  and  must  be  “redevel- 
oped” for  each  buy.  Some  commercial  manufacturers  advertise  and  take 
orders  for  equipment  still  under  design;  the  design  may  be  canceled  due  to 
lack  of  interest  or  altered  by  a large  user  with  substantially  different  needs. 

4.  Candidates  not  conforming  to  the  Buy  American  Act  (41  USC  lOB(bl)  un- 
less there  is  no  domestic  candidate  available,  the  item  is  for  use  in  the  coun- 
try of  origin,  or  the  item  is  a Canadian  Military  Supply  (such  supplies  are 
exempted  from  the  Buy  American  Act;  see  ASPR  6-1 03.5(a)  and  NPD 
6-103.5). 

5.  Candidates  obviously  not  meeting  project  cost  constraints. 


If  a substantial  number  of  candidates  remain,  the  list  can  be  further  narrowed  by 
requiring  further  compliance  to  soft  technical  requirements.  Also,  it  is  desirable  to  elim- 
inate unreliable  manufacturers;  however,  this  action  must  be  in  compliance  with  ASPR 
Section  1,  Part  6 (1  July  1074). 

Prepare  a priority  list  of  candidates  based  upon  their  conformance  to  all  acquisition 
requirements,  their  success  as  products  (talk  to  users  and  distributors),  their  apparent 
ability  to  perform  in  application.,  closely  related  to  the  intended  use,  the  availability  of 
long-term  warranties  on  them,  and  other  general  indicators  of  ruggedness,  reliability,  avail- 
ability, and  suitability. 


CONDUCTING  VISUAL  INSPECTIONS 

Candidate  equipments  should  be  inspected  visually  for  workmanship,  enclosure 
effectiveness,  human  engineering,  safety  design,  maintainability,  operability,  and  thought- 
fulness and  vintage  of  design.  Major  and  minor  discrepancies  should  be  noted.  Major 
discrepancies  are  cause  for  rejection  if  they  cannot  be  corrected  through  simple  modifi- 
cations. An  archaic  design  may  be  cause  for  rejection  unless  the  manufacturer  can  guaran- 
tee supply  parts  support.  Too  new  a design  (experimental  or  unproved)  should  be  rejected 
unless  the  manufacturer  is  willing  to  offer  a long-term  warranty. 


REVIEWING  DATA 

All  available  technical  manuals,  instructions,  schematics,  and  other  equipment  data 
should  be  reviewed  to  determine  whether  they  are  adequate  to  support  the  project  needs. 
Technical  manuals  should  meet  the  standards  of  Mll.-M-72‘>X,  Manuals,  Technical.  Conuncr 
rial  Equipment.  Obtain  copies  of  schematics,  parts  lists,  and  technical  manuals  wherever 
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possible.  Review  the  schematics  for  apparent  capability  of  meeting  the  specification  and 
interface  requirements.  Review  the  parts  list  to  determine  the  reliability  and  availability 
of  the  parts.  Review  the  operations  and  maintenance  procedures  to  determine  their 
clarity  and  completeness. 

Using  a combination  of  hardware  and  documentation,  evaluate  the  following  items: 
Net  input  power/volume  (an  indication  of  heat  buildup). 

Operating  temperature  (an  indication  of  component  design  limitations  and  humidity 
resistance). 

Cooling  capability  /net  input  power  (an  indication  of  cooling  effectiveness). 

Internal  voltage  levels  (another  indication  of  humidity  resistance). 

Equipment  weight,  volume,  shape,  and  mounting  provisions  (indicators  of  shock 
and  vibration  resistance,  and  interface  or  installation  problems). 

Manufacturer’s  claimed  environmental  specifications,  if  any. 

Component  weight,  volume,  shape,  and  mounting  provisions  (further  indicators  of 
shock  and  vibration  resistance). 

Electronic  component  count  (an  indication  of  reliability,  maintainability,  and  cost 
of  logistics  support).  (Mechanical  parts  also  fail,  but  such  failures  tend  to  be 
negligible  in  electronics  equipment.  A relative  parts  count  between  competing 
equipments  cannot  tell  as  much  as  a weighted  parts  count  based  on  parts  failure 
rates.  However,  all  that  is  necessary  the  first  time  through  this  step  is  a rough 
approximation.) 

CONTACTING  MANUFACTURERS.  DEALERS.  AND  USERS 

Sales  brochures  and  data  sheets  often  simplify  or  exaggerate  specifications.  They 
may  make  claims  that  cannot  be  substantiated.  Performance  specifications  may  apply 
only  to  limited  circumstances  and  ideal  conditions,  or  may  be  valid  only  with  special 
options.  Certain  specified  performance  levels  may  not  be  realizable  simultaneously. 

• Contact  the  manufacturer’s  marketing  and  engineering  departments.  Discussions 
with  technical  experts  from  the  company  may  reveal  hidden  defects  or  additional 
capabilities  not  apparent  from  sales  literature.  New  products  may  be  in  produc- 
tion. Perhaps  variations  of  the  advertised  equipment  or  special  options  are 
available. 

• Ask  the  manufacturer  to  explain  specifically  why  his  product  is  better  than  the 
others  under  consideration.  An  alert  manufacturer  knows  his  competition  and 
is  eager  to  show  off  his  product’s  advantages.  Such  comparative  information 
will  point  out  subtle  problems  not  readily  visible  in  product  sales  literature  and 
not  likely  to  be  raised  by  a manufacturer  with  regard  to  his  own  product. 

• Sift  through  the  comparisons  furnished  by  all  companies.  Give  each  manu- 
facturer a chance  to  defend  his  equipment  or  to  offer  solutions  to  any  serious 
problems  pointed  out  by  competitors,  without  identifying  the  source  of  your 
information. 

• Obtain  the  names  of  dealers  who  supply  the  product.  Solicit  dealer  comments 
on  the  suitability  of  his  equipment  for  the  application,  his  experience  with  the 
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manufacturer,  and  customer  comments  or  complaints  he  has  received.  Obtain, 
if  possible  (from  dealer  or  manufacturer),  a history  of  the  item's  usage  and  fail- 
ure, and  a list  of  purchasers. 

• Request  information  from  purchasers  regarding  performance,  problems,  or 
failures  versus  usage.  Determine  whether  the  purchasers’  mounting  locations, 
operational  requirements,  and  environments  are  comparable  to  those  of  the 
intended  application  so  that  these  comments  can  be  effectively  evaluated. 


OBTAINING  UNITS  FOR  TEST 

Usually,  it  is  not  necessary  to  actually  acquire  equipments  until  the  performance 
and  environment  tests  are  conducted.  The  equipments  can  often  be  viewed  and  inspected 
at  a user’s  facility  or  the  manufacturer’s  plant.  However,  the  equipments  must  be  “in 
hand’’  for  testing. 

If  the  procurement  requirements  offer  the  prospective  suppliers  the  promise  of 
reasonably  large  sales,  there  will  be  sufficient  incentive  to  the  supplier  to  make  a unit  or 
two  available  (on  loan)  for  testing.  If  a formal  “sources  sought’’  has  been  used  to  find 
candidates,  and  if  it  appears  reasonably  certain  one  or  more  candidates  will  be  acceptable, 
a bid  sample  can  be  requested.  In  either  case,  equipment  is  acquired  for  test  at  no  cost 
to  the  government.  However,  the  supplier  still  owns  the  equipment,  and  the  government 
cannot  test  modifications  or  test  to  extreme  environments  without  the  supplier’s  written 
permission.  Also,  the  supplier  has  the  right  to  witness  the  screening  process;  this  can 
present  problems  of  security  in  keeping  multiple  suppliers  away  from  a competitor’s 
equipment. 

The  alternative  method  is  to  buy  sample  equipments  of  the  leading  candidates. 
This  presents  the  problem  of  funding  the  procurement  if  a large  number  of  candidates 
or  expensive  equipments  are  involved.  If  every  fully  preliminarily  qualified  candidate  is 
not  included,  the  screen’s  effectiveness  in  gathering  data  for  procurement  is  defeated. 

Tight  specifications  and  accurate  and  complete  records  are  needed  to  keep  the  number  of 
candidates  within  reasonable  limits;  these  same  items  are  required  to  guard  against 
challenges  resulting  from  procurements  based  on  the  screen  results.  This  disadvantage  is 
usually  balanced  by  the  advantages  of  the  government’s  owning  the  equipment;  early 
budgetary  planning  is  needed  to  ensure  adequate  funding  for  the  test  cycle. 


CONDUCTING  PERFORMANCE  AND  ENVIRONMENTAL  TESTS 

Screening  tests  should  normally  include  the  following  steps: 

1.  Initial  performance  test  to  the  manufacturer's  specifications;  where  several 
parameters  interreact,  choose  "worst  case"  test  conditions. 

2.  Testing  to  project  specifications  including  environmental  limits. 

3.  Burn-in  for  100  hours. 

4.  Combined-environment  test  to  specified  environmental  limits  for  temperature, 
humidity,  vibration,  and  electrical  transient  response  as  a minimum.  Add 
stresses  which  may  he  encountered  in  use  from  environmental  cycling,  repe- 
titive shock,  and  1MI  as  appropriate. 
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It  is  desirable  to  test  at  least  three  units  of  each  candidate. 

Normally  these  steps  conclude  the  screening  tests:  the  entire  test  cycle  can  be  com- 
pleted in  as  little  as  3 weeks  for  simple  equipments.  The  combined-environment  test  (CET) 
requires  120  hours.  The  total  test  time  may  be  extended  for  more-complex  equipments,  by 
repair  time  should  failures  occur,  or  by  problems  in  scheduling  the  test  apparatus. 

The  screening  tests  measure  the  design  suitability  of  the  equipment  in  the  intended 
application;  they  do  not  directly  measure  reliability.  Reliability  predictions  in  conjunction 
with  the  screening  process  are  normally  sufficiently  accurate  lor  planning  purposes;  howev- 
er. reliability  tests  may  be  required  if  a particular  minimum  acceptable  reliability  is  essential. 
The  cost  of  reliability  testing  can  be  reduced  markedly  by  combining  the  reliability  test  and 
the  CET,  using  multiple  equipments  and  extending  the  test  time  as  necessary  to  accumulate 
a sufficiently  high  equipment  “on”  time. 

The  Screening  Acceptance  Criteria  are  as  follows: 

1.  No  repeatable  failures  for  any  single  test  sequence.  (Example:  A unit  which 
fails  a vibration  test  twice  is  not  acceptable.) 

2.  No  pattern  failures  (as  defined  in  MIL- STD-781 ). 

3.  Measured  MTBE  meets  or  exceeds  project  requirements  see  table  Xl-1 ) with  an 
acceptable  confidence.  High  MTBE  requirements  may  dictate  using  failure-free 
trial  acceptance  criteria  (see  table  Xi-2).  All  units  must  pass  failure-free  within 
3 trials  of  the  CET  phase. 

4.  Predicted  performance  remains  within  required  limits  when  weighted  by  quali- 
ty factors.  (See  PREDICTING  PERFORMANCE.) 
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TABLE  XI-:.  FAILURE-FREE  ACCEPTANCE  CRITERIA  FOR  C FT  TRIALS. 


n = lot  size 

t = Cfc'T  test  time  (normally  120  hi 
0 = specified  MTBF  (must  be  t X 4 or  greater) 

N = number  of  trials  allowed  to  pass  the  entire  lot  (round  up  to  the  nearest  whole 
number) 

(N-n  = number  of  allowable  failure  trials) 

For  intermediate  ranges  of  ft,  a trial  length  can  be  assumed  at  1 CFT  cycle  (24  hours).  For  0 greater 
than  1 200  hours,  additional  test  time  should  be  added,  if  possible,  in  24-hour-cycle  increments.  It  is 
desirable  for  the  lot  size  to  be  at  least  3 but  less  than  1 2.  A maximum  of  two  failure  trials  is  allowed 
for  any  single  equipment. 
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EVALUATING  THE  RESULTS 

Ideally,  multiple  candidates  will  be  qualified  by  the  screen  without  reservation. 

The  screen  results  can  then  be  used  to  procure  equipments  for  service  use  (see  chapter  VI). 
However,  elements  of  the  intended  application  and  of  the  equipment  design  generally  do 
not  mesh  perfectly,  so  the  screening  decision  is  not  clearly  determined.  An  economic 
analysis  is  normally  required  to  determine  the  acceptability  of  each  candidate.  By  adapt- 
ing the  system  life-cycle  cost  model,  a sufficiently  accurate  analysis  can  be  completed. 
Subjective  evaluations  will  be  necessary  to  develop  some  of  the  inputs  to  the  model;  how- 
ever, maintenance  and  logistic  support  data  can  be  improved  in  quality  by  using  informa- 
tion supplied  by  current  users  and  obtained  from  experiences  during  the  test  cycle.  The 
data  are  further  fixed  if  warranties  are  available.  In  general,  reliability/failure  rate  data 
are  most  difficult  to  obtain. 

The  reliability  of  established  products  is  already  designed  and  built  in;  talks  with 
the  manufacturer,  observing  his  production  quality  assurance,  and  user  acceptance  of  the 
product  in  similar  application  environments  combine  to  give  qualitative  assurance  that 
the  reliability  of  the  equipment  is  adequate.  However,  quantitative  values  are  required  for 
further  project  planning.  Reliability  predictions  can  supply  sufficiently  accurate  values 
when  weighted  by  the  screening  data. 

It  is  important  not  to  overlook  the  five  major  issues  which  must  be  resolved  for 
any  acquisition  (see  chapter  VI).  These  qualitative  decisions  may  exclude  one  or  more 
candidates.  The  purpose  of  the  screen  is  to  avoid  development  wherever  possible  and 
economical;  however,  the  satisfaction  of  the  operational  requirements  must  be  the  para- 
mount goal  of  the  project. 


PREDICTING  PERFORMANCE 

There  are  three  elements  of  field  performance  which  are  important  to  program 
planning  and  system  implementation: 

Operational  performance 
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Field  reliability 

Useful  service  life 

The  operational  performance  is  essential  because  the  equipment  is  not  worth  procur- 
ing without  it.  To  predict  operational  performance,  the  figure-of-merit  (FOM)  discussed 
under  PERFORMANCE  PARAMETERS  in  chapter  IV  should  be  utilized.  Taking  the  values 
measured  for  the  critical  performance  factors  and  applying  the  FOM  model,  a FOM  can  be 
computed  for  each  unit  tested;  this  number  becomes  FOM(m).  L is  the  limit  computed 
from  the  minimum  specification  parameters  using  the  FOM  model.  Find  the  expected 
service  quantity  in  table  A-2  of  MIL-STD-414  and  convert  the  inspection  level  code  to  a 
sample  size  using  table  B-3.  Using  the  computed  QL  resulting  from  the  table  XI-3A  proce- 
dure and  the  sample  size  corresponding  to  the  expected  service  quantity,  table  B-5  of  M1L- 
STD-414  will  give  a lot  percent  defective  which  equals  the  percent  risk  of  an  unacceptable 
unit  going  into  service  use  without  further  screening.  Generally,  a 1-percent  risk  of  this 
nature  is  very  acceptable. 

The  most  reliable  method  of  predicting  field  reliability  is  through  reliability  testing. 
MIL-STD-781  provides  test  plans  with  various  degrees  of  statistical  accuracy  ranging  from 
risks  of  10  percent  to  50  percent  or  more.  (See  fig.  Xl-1 .)  However,  the  accuracy  of  these 
predictions  is  determined  by  the  accuracy  of  the  specified  test  environment  in  simulating 
the  field  environment  including  input/output  conditions  caused  by  failures  in  interfacing 
equipment  and  including  all  failure  conditions  during  the  test.  Even  a properly  run  test  will 
tend  to  overestimate  field  reliability  significantly  simply  because  the  tests  measure  MTBF 
as  a design  characteristic  rather  than  MTBCM.  which  includes  maintenance-induced  failures. 


EXPECTED  COST  (EXPECTED  TEST  TIME  (HOURS)  NUMBER  OF  UNITS  UNDER  TEST) 
(COST /HOUR)  + (EXPECTED  NUMBER  FAILURES  ■ COST/F  Al  LURE) 


Figure  Xl-1 . Risk  as  a function  of  tesi  time. 


XI -4 


false  removals,  and  pseudo  failures  and  also  quality-  and  workmanship-related  failures.  Com- 
pute the  workmanship  factor  in  accordance  with  table  XI-3B.  Compute  the  relationship 
between  "inherent  MTBF"  and  "base  MTBCM"  using  table  X1-3C,  but  temper  the  results 
with  engineering  judgment.  The  predicted  field  reliability  is  the  product  of  the  predicted 
MTBF  from  the  test  results,  the  quality  level  (from  table  X1-3A),  the  workmanship  factor, 
and  the  MTBCM,  MTBF  ratio.  If  test  results  are  not  adequate  to  give  a valid  reliability  pre- 
diction. use  the  parts  stress  analysis  prediction  procedure  in  part  2 of  M1L-HDBK-2 1 7 to 
establish  a baseline.  Determine  the  parts  count  prediction  in  accordance  with  part  3 of 
M1L-11DBK-21 7,  and  establish  the  parts  stress  analysis  prediction  to  parts  count  prediction 
ratio  (this  ratio  is  a measure  of  the  derating  factor  used  in  the  design).  Multiply  the  stress 
analysis  prediction  by  the  prediction  ratio  to  establish  an  approximate  equivalent  to  a pre- 
diction based  upon  test  results,  and  predict  field  reliability  as  above. 

The  useful  service  life  of  an  equipment  is  an  important  but  frequently  neglected 
element  of  field  performance.  If  the  useful  service  life  is  exceeded,  the  operation  and  sup- 
port costs  will  rise  markedly,  often  tripling  annually.  There  are  many  factors  which  affect 
service  life  such  as  usage  environment,  design  technology  . changing  user  requirements,  and 
the  time-to-wearout  characteristic.  If  the  equipment  was  designed  for  a usage  environment 
comparable  to  the  actual  one  and  assuming  that  the  design  technology  and  user  require- 
ments do  not  cause  obsolescence,  the  time-to-wearout  becomes  the  primary  determinant  of 
the  useful  service  life. 

The  factors  which  influence  time-to-wearout  include  design  features  and  production 
quality  and  workmanship.  Many  of  these  factors  are  incorporated  in  the  prediction  of  field 
reliability  as  the  time-to-wearout  is  a function  of  1 1 ) the  number  of  repairs  required  and  ( 2 ) 
the  deterioration  caused  by  the  repair  action.  I he  time-to-wearout  can.  therefore,  be  ex- 
pressed as  a product  of  the  field  reliability  and  a repair  factor.  Die  repair  factor  can  be 
calculated  in  accordance  with  table  XI-31):  it  includes  repair  features,  training  levels,  and 
support  system  characteristics  which  are  determined  by  the  design  of  the  equipment  and  of 
the  support  system.  The  results  will  be  stated  in  hours  and  must  usually  be  converted  to 
years  by  dividing  by  the  expected  usage  (operating  hours  per  year). 

Experience  has  shown  that  there  are  essentially  three  distinct  grades  of  equipment: 
commercial,  industrial,  and  military.  Each  grade,  when  used  in  military  applications,  exhib- 
its useful  service  life  characteristics  as  follows: 

Commercial  1-2  years  and  as  high  as  5 years 

Industrial  5-b  years  and  as  high  as  10  years 

Military  10-1  2 years  and  as  high  as  30  years 

In  considering  service  life,  cost  and  operational  performance  should  be  taken  into  account. 
An  equipment  with  a long  expected  service  life  may  not  exhibit  the  necessary  performance 
to  meet  expected  future  requirements.  A short-lived  equipment  may  be  so  inexpensive  in 
comparison  to  a long-lived  equivalent  that  lower  life-cycle  costs  will  resul)  from  multiple 
procurements.  However,  multiple  procurements  of  small  quantities  of  peculiar  items  will 
not  he  cost-effective.  If  the  technology  or  user  requirement  factors  are  very  unstable,  a 
relatively  short  service  hie  equipment  may  be  indicated;  conversely  , an  equipment  inextrica- 
bly associated  with  a ship  or  air  platform  must  serve  for  the  life  of  the  platform. 

Despite  the  deterministic  methods  ot  predicting  these  Held  performance  factors,  an 
equipment  will  not  begin  to  meet  predictions  without  thorough  planning  and  execution  ol 
the  acquisition  and  support  steps.  The  II  N tasks  must  be  completed  properly  to  make  the 
assumptions  valid  which  underpin  these  prediction  methods.  Of  the  predictions,  operational 
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performance  will  tend  to  be  most  accurate  (usually  2-place  accuracy  is  provided),  and  serv- 
ice lite  will  tend  to  be  least  accurate  (±3 O'/r).  Although  not  precisejn  absolute  terms,  the 
predictions  will  tend  to  provide  accurate  comparisons  between  afternatives.  No  prediction 
can  completely  replace  experienced  engineering  judgment,  but  the  methods  presented  here 
can  be  useful  tools  in  program  planning  and  system  implementation. 


TABLE  Xl-3.  PERFORMANCE  PREDICTION  CALCULATIONS. 


A.  Quality  Level  Calculation  For  the  i1'1  Unit 


Xj  = FOM(m)  - L 


Y 


Calculate  the  mean  = 


i=l 


-=  M 


n 

Y 


2 V- Pr- 


e- 


calculate the  variance  = 


n-1 


= V 


Calculate  the  standard  deviation  = S = 

M 

Calculate  the  quality  level  = QL  = — 


(derived  front  MIL-STD414) 


B.  Workmanship  Factor  Calculation 


D = number  of  performance-relevant  major  defects  cited  by  workmanship  inspection  in  accordance 
with  an  established  standard  such  as  MIL-STD-252 

N = number  of  units  inspected 

P = number  of  projected  service  units 

C = level  of  complexity  of  unit  inspected  (per  chapter  VI) 


WF  = 


( I - p ) or  . whichever  is  greater 


In  (C  + 1 .5)  - log  N - .82  (derjved  (rom  MIL-STD-105) 


C.  MTBCM/MTBF  Ratio  Calculation 


Inspect  the  design  t\>r  the  characteristics  listed  and  sum  the  indicated  values. 


Characteristic 

Yes 

No 

Notes 

1 . Failure  alarms 

10 

■N 

1.2 

2.  Built-in  diagnostic  test 

1 

1 

3.  Test  points  and  adjustments  are  easily  accessible  and 
protected  from  short  circuits  to  ground  and  to  sur- 
rounding circuitry 

\ 

3 



Notes  I.  multiply  by  fraction  ol  equipment  functions  covered 
2.  double  lor  redundant  functions 
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TABLE  XI-3.  (Continued). 


A = sum  of  indicated  characteristic  values 
B = fraction  of  functions  with  redundancy  built  in 
MTBCM  = A_  2 

MTBF  60  (derived  empirically) 

D.  Repair  Factor  Calculation 

Evaluate  the  equipment  in  accordance  with  the  values  below  to  establish  the  maintenance  life  factor. 


1 . Ease  of  replacement  factor 

plugged  in 
screwed  down 
soldered  (to  terminals) 
soldered  ( to  PC  board) 
wire  wrap/crimp/wire  weld 
potted  or  coated 

60 

48 

40 

32 

1 6 
“> 

2.  Level  of  repair 

Organization 

4 

Ihtermediate 

8 

Depot 

l 

3.  Technician  experience  level 

unskilled/E-3  or  below 

semiskilled/E-4  or  E-5 

8 

skilled/E-6  or  above 

16 

Maintenance  life  factor  (MLF)  = sum  of  values  from  (1,2.31  above  -f  30 

Supply  support  factor  (SSF)  = ratio  of  preferred  standard  parts  (per  M1L-STD-242 ) to  total  number 

of  parts 

RF  = I + (In  C)(l. 82)'  1C  (MLF) (SSF) 

C = complexity  (per  chapter  VI) 


(derived  empirically  I 


XII.  SYSTEM/DEVELOPMENT  SPECIFICATION 


XII  SYSTEM  DEVELOPMI  NT 
SPECIFICATION 


XU.  SYSTEM/DEVELOPMENT  SPECIFICATION 


Prograin-peculiar  system,  development,  and  product  specifications  am  required  for 
system/equipment  items  undergoing  engineering  or  operational  development  at  government 
expense.  The  top-level  specification  the  first  to  be  prepared  and  the  one  of  concern  to 
this  section  is  intended  to  establish  the  functional  configuration  identification  of  the 
item  being  developed. 

According  to  MIL-STIMhO  there  are  three  types  of  specifications  the  contents  of 
which  identify  the  functional  configuration  of  an  item  the  Type  A “system”  specifica- 
tion and  the  Types  131  and  B2  “development”  specifications.  Which  to  use  depends  on 
the  complexity  of  the  item.  The  Type  A specification  is  necessary  for  a system  comprising 
“subsystems,  assemblies  (or  sets),  skills,  and  techniques  capable  of  performing  and/or  sup- 
porting an  operational  lor  nonoperational)  role  ...  to  the  degree  it  can  be  considered  a 
self-sufficient  item  in  its  intended  . . . environment”  (MIL-STD^480).  In  addition  to 
requirements  common  to  the  functional  identification  of  all  hardware  developments,  this 
specification  would  require  definition  of  the  extent  to  which  the  missions  of  the  system 
affect  design  requirements,  current  and  potential  enemy  threats  to  the  system,  operational 
and  organizational  concepts  which  constrain  design  and  operation,  nuclear  control  require- 
ments. and  coverage  of  system  effectiveness  models. 

A Type  131  specification  is  applicable  to  a “prime”  item.  This  item  does  not  have 
the  complexity  of  a system,  but  in  order  to  define  it  properly  in  the  specification  it 
generally  would  be  complex  enough  to  have  to  include  (1)  functional  flow  diagrams  to  the 
level  required  to  identify  its  several  essential  functions;  (2)  functional  and  physical  inter- 
laces between  it  and  other  items  and  between  the  major  components  within  itself;  (3)  a 
major  component  list:  and  (4)  lists  of  government-furnished  and  -loaned  property  incor- 
porated In  the  item.  In  addition,  this  item  will  require  provisioning  actions,  operation  and 
maintenance  manuals,  and  quality  conformance  inspection  of  each  unit  of  the  item. 

rite  functional  configuration  of  a “critical”  item  one  below  the  complexity  of 
i prime  item  can  be  described  adequately  by  a Type  132  specification. 

When  a s\  stem  is  covered  by  a Type  A specification  and  additional  specification 
el  it  major  turn  tional  areas  is  necessary  to  completely  identify  system  functional  config- 
uration. I > pcs  Bl  and  B2  specifications  also  are  used  as  second-level  development  speci- 
fications to  describe  the  “allocated"  functions  of  subsystems  and  other  major  items  of  the 
system,  flic  criteria  of  complexity  described  above  apply  to  which  Type  13  specifications 
to  use  lor  allocated  I unctional-con figuration  identification. 

Content  and  format  of  Types  A.  Bl,  and  B2  specifications  are  indicated  in  appen- 
dices 1 11  and  111.  respectively,  of  M1L-STD-440  (also  see  table  at  end  of  this  section  for 
outline  of  contents). 

Besides  the  t\pe  ot  specification,  which  “form”  to  use  must  be  considered. 

Mil  S-834‘H)  specifies  the  use  of  four  different  forms  which  differ  in  the  degree  of  control 
they  provide  over  format  and  content.  They  are  identified  as  “Form  la.  Form  lb.  Form 
2.  and  Form  3."  Fhe  degree  of  control  required  depends  on  how  refined  the  specification 
must  be  to  satisfy  government  needs  for  the  particular  development. 

form  la  specifications  are  prepared  to  rigid  military  standards  where  extensive 
control  of  paragraph  content  and  format  is  necessary.  They  must  conform  to  M1F-SID- 
4‘>0  in  all  respects,  including  paragraph  numbering  and  titling  and  subject  coverage  as 
specified  m the  appendices  of  MIL-STl)-4'>0. 
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Form  lb  specifications  also  are  prepared  to  military  standards,  but  where  limited 
format  control  is  allowable  along  with  maximum  content  control.  They  can  be-  prepared 
according  to  the  requirements  of  Defense  Standardization  Manual  4120.3M  or  MIL-SI  D- 
490.  If  M1L-STD-490  is  followed,  the  strict  paragraph  sequencing,  numbering,  and  titling 
of  its  appendices  are  not  obligatory,  although  the  six-section  format  is  mandatory. 

Form  2 specifications  are  prepared  to  commercial  standards  with  supplemental 
military  requirements  when  such  specifications  will  be  acceptable  lor  the  government  s 
intended  use  (possibly  with  minor  change)  and  offer  a price  or  delivery  advantage  over 
Form  1 specifications.  The  supplemental  military  requirements  are  (1)  characteristics 
must  be  specified  to  the  degree  necessary  to  allow  eventual  procurement  and  delivery  of 
an  item  that  meets  all  government  requirements;  (2)  quality  assurance  provisions  must  be 
included  to  assure  the  meeting  of  these  requirements;  (3)  symbols,  reference  designations, 
codes,  abbreviations,  etc,  must  be  to  military  standards  or  be  fully  explained;  and  (4) 
the  commercial  standard  to  which  the  specification  is  prepared  must  be  furnished  or  be 
referenced  and  available. 

Form  3 specifications  are  prepared  merely  in  accordance  with  the  contractor's 
agency’s  normal  practices,  but  must  satisfy  the  intended  use  of  the  document. 

Although  strict  adherence  to  the  appendices  ol  M1L-STD-490  is  not  mandatory 
for  Forms  lb,  2,  and  3 specifications,  use  of  the  appendices  as  a guide  is  encouraged  to 
assure  adequate  coverage  of  all  requirements  that  may  be  necessary  lor  the  particular 
development.  (Refer  to  the  table  at  the  end  ol  this  section.) 

Types  A.  Bl,  and  B2  specifications  establish  a functional  configuration  baseline 
against  which  changes  are  evaluated  and  made  as  design  development  progresses.  When 
design  changes  are  approved  and  incorporated,  the  documented  baseline  is  updated  accord- 
ingly to  ensure  continual  correspondence  between  the  item’s  actual  configuration  and  the 
documentation  which  describes  it.  This  baseline  identification  serves  throughout  the  life 
cycle  of  the  item  as  a description  of  required  functional  characteristics. 

Functional  baselines  should  be  specified  flexibly  to  avoid  undesired,  premature  com- 
mitments to  detailed  requirements  that  would  be  difficult  and/or  expensive  to  change.  In 
the  case  especially  of  major  system  developments,  the  initial  baseline  identification  could 
well  be  a preliminary  system  description  of  a range  of  proposed  broad  performance  para- 
meters or  characteristics  which  then  could  be  used  to  facilitate  the  evaluation  ot  alterna- 
tive design  approaches  as  performance-cost  tradeoffs  are  made. 

A flexible  functional  baseline  is  a good  start  toward  later  establishment  of  product 
specifications  which  are  performance  oriented  rather  than  design  oriented  “product- 
function”  specifications  rather  than  “product  fabrication”  specifications.  Various  studies 
and  current  directives  emphasize  the  preference  ol  the  former  over  the  latter  except  tor 
developments  where  materials,  individual  parts,  and/or  internal  configuration  must  be 
controlled  because  of  particular  military  needs,  l’ertormance-type  specifications  (ie.  form, 
fit,  function  specifications)  which  include  environmental  and  interface  requirements,  as 
opposed  to  detailed  design  specifications  requiring  particular  parts,  processes,  materials, 
and  internal  configuration,  assure  multiple-source  procurement.  Also,  by  including  stand- 
ardized mechanical,  electrical,  and  environmental  interface  requirements  in  performance- 
type  specifications  for  items  applicable  to  common  operational  functions,  flic  interchange- 
ability  of  similar  equipments  can  be  ensured.  This  results  in  ready  replacement  of  old 
designs  by  new-generation  equipment  as  well  as  enhancing  design  and  price  competition 
among  contractors.  However,  the  other  side  of  the  coin  proliferation  of  detailed  designs 
must  be  considered,  too,  tor  optimum  end  results,  lo  minimize  variety  and  to  control 
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design  features  which  pertain  to  interchangeability,  compatibility,  reliability,  and  maintain- 
ability, it  may  be  necessary  to  include  some  detailed  design  requirements  in  performance- 
type  specifications. 

In  exploring  and  establishing  system/equipment  requirements  for  specification,  cost, 
quantity,  and  schedule  constraints  should  be  given  equal  status  to  performance  and  physical 
characteristics.  Cost  factors  should  be  introduced  as  early  in  the  conceptual  design  and 
planning  phases  as  possible,  and  formal  requirements  in  final  form  should  be  specified 
and  issued  only  after  several  iterations  of  cost-performance  estimates.  Not  only  should 
first  cost  (design  to  cost)  be  weighed  carefully,  the  costs  of  maintenance,  supply,  and 
training  — along  with  the  tradeoff  values  of  cost,  reliability,  performance,  maintainability, 
and  efficiency  - should  be  carefully  considered. 

The  following  features  can  increase  cost: 

Needless  refinement 
Expensive  finishes 
Unreasonable  tolerances 
Critical  materials 
Restrictive  processes 
Overemphasis  on  appearance 

Overlong  life  expectance  as  related  to  intended  use 
Overprotection  against  failure 
Unreasonable  and  excessive  inspection 
Unrealistic  reliability 

A special  case  exists  when  establishing  requirements  to  be  specified  for  items  which 
are  to  be  supported  initially  by  contractor  long-term  warranties.  Here,  reliability,  main- 
tainability, and  initial  provisioning  are  not  the  major  concerns  they  otherwise  would  be. 
These  requirements  may  be  relaxed  along  with  their  quality  assurance  provisions  when 
there  are  adequate  contractual  warranty  provisions  which  make  it  profitable  for  the  con- 
tractor to  design  and  build  highly  reliable  and  maintainable  equipment. 

Regardless  of  the  scope  of  requirements  in  a specification,  each  requirement  should 
be  stated  to  present  exactly  what  the  government  wants.  Requirements  should  be  so 
worded  as  to  provide  a definite  basis  for  rejection  when  testing  and  inspection  reveal  un- 
suitability. Also,  care  should  he  exercised  to  avoid  unrealistic  requirements  and  those 
which  conflict  with  referenced  documents.  If  requirements  are  not  absolutely  clear  and 
definite,  bidders  will  not  know  exactly  what  they  are  to  furnish  in  order  to  make  a respon- 
sible bid. 

The  contents  of  a specification  pertain  directly  and  only  to  the  item  to  be  devel- 
oped. Thus,  the  following  activities  of  a development  program  are  not  covered  by  a 
specification,  but  rather  by  the  statement  of  work  of  the  contract: 

Configuration  management 
Integrated  logistics  support 
Safety  program 
Human  factors  program 
Training  program 
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Level  of  repair 

Acquisition  management 

Supply  support 

Contractor  services 

Quality  program 

Reliability  program 

Maintainability  program 

Planned  maintenance  subsystem 

Test  point  program 

Calibration  and  instrumentation 

Electromagnetic  compatibility  program 

Special  test  programs 

In  addition,  the  specification  must  not  contain  requirements  for  the  delivery'  of 
data.  Data  to  be  delivered  under  the  contract  can  only  be  ordered  by  the  Contract  Data 
Requirements  List,  DD  Form  1423.  The  form  and  content  of  each  line  item  of  data  on 
the  DD  Form  1423  arc  required  to  be  specified  by  a Data  Item  Description,  DD  Form 
1664,  which  is  attached  to  and  becomes  a part  of  the  DD  Form  1423.  Wh$n  data  require- 
ments are  contained  in  a DD  Form  1664,  such  requirements  must  not  appear  in  the  speci- 
fication except  that  deliverable  data  can  be  identified  in  Section  6.  Notes,  of  the  specifica- 
tion. This  identification  should  include  the  corresponding  DD  Form  1664  numbers.  An 
exception  allowing  data  requirements  to  be  described  directly  in  a specification  occurs 
when  it  is  impractical  to  separate  such  description  from  its  context.  An  example  of  this 
would  be  data  requirements  covering  quality  assurance  provisions. 

Following  is  a table  of  paragraph  requirements  for  functional-configuration- 
identification  specifications  prepared  to  Form  la  requirements.  The  paragraph  titles  and 
numbers  conform  to  Appendices  I,  H,  and  111  of  MIL-STD-490.  The  table  can  be  used  as 
a checklist/guide  during  specification  preparation;  content  requirements  for  each  paragraph 
are  presented  under  the  respective  paragraph  number  of  the  applicable  appendix  in  MIL- 
STD-490. 
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MIL-STD490.  SPECIFICATION  PRACTICES 
PARAGRAPH  REQUIREMENTS  FOR  SYSTEM/DEVELOPMENT  SPECIFICATIONS 

PARAGRAPH  TITLE  SPECIFICATION  TYPE  AND  PARAGRAPH  NO 


Bl 

B2 

A 

Prime  Item 

Critical  Item 

System 

Development 

Development 

(Append.  1) 

(Append.  II) 

(Append.  Ill) 

SCOPE 

1 

1 

1 

Scope  statement 

1.1 

1.1 

1.1 

APPLICABLE  DOCUMENTS 

2 

2 

2 

Government  documents 

2.1 

2.1 

2.1 

Non-Government  documents 

->  •> 

t 2 

■ ->  2 

REQUIREMENTS 

3 

3 

3 

System  definition 

3.1 

Item  definition 

3.1 

3.1 

General  description 

3.1.1 

Item  diagrams 

3.1.1 

Missions 

3.1.2 

Interface  definition 

3.1.5 

3.1.2 

Threat 

3.1.3 

Major  component  list 

3.1.3 

System  diagrams 

3.1.4 

Government  furnished  property  list 

3.1.6 

3.1.4 

Government  loaned  property  list 

3.1.5 

Operational  and  organizational  concepts 

3.1.7 

Characteristics 

3.2 

3.2 

3.2 

Performance  characteristics 

3.2.1 

3.2.1 

3.2.1 

Physical  characteristics 

3.2.2 

3.2.2 

3.2.2 

Reliability 

3.2.3 

3.2.3 

3.2.3 

Maintainability 

3.2.4 

3.2.4 

3.2.4 

Availability 

3.2.5 

Environmental  conditions 

3.2.7 

3.2.5 

3.2.5 

System  effectiveness  models 

3.2.6 

Transportability 

3.2.9 

3.2.6 

3.2.6 

Nuclear  control  requirements 

3.2.8 

Design  and  construction 

3.3 

3.3 

3.3 

Materials,  processes  and  parts 

3.3.1 

3.3.1 

3.3.1 

Electromagnetic  radiation 

3.3.2 

3.3.2 

3.3.2 

Nameplates  and  product  marking 

3.3.3 

3.3.3 

3.3.3 

Workmanship 

3.3.4 

3.3.4 

3.3.4 

Interchangeability 

3.3.5 

3.3.5 

3.3.5 

Safety 

3.3.6 

3.3.6 

3.3.6 

Human  performance/human  engineering 

3.3  7 

3.3.7 

3.3.7 

Documentation 

3.4 

3.4 

3.4 

Logistics 

3.5 

3.5 

3.5 

Maintenance  (logistics) 

3.5.1 

3.5.1 

3.5.1 

Supply 

3.5.2 

3.5.2 

3.5.2 

Facilities  and  facility  equipment 
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XIII.  DECISION  TO  BUILD, 
BUY,  OR  MODIFY 


XIII.  DECISION  TO  BUILD.  BUY,  OR  MODIFY 


Commercial  enterprises  base  their  make-or-buy  decisions  on  economics.  The  mili- 
tary frequently  bases  the  decision  on  expediency.  The  design  and  procurement  approach 
should  result  in  the  provision  of  the  required  equipment  characteristics  to  the  Navy  at 
lowest  system  life  costs.  This  is  done  through  the  use  of  life-cycle  cost  (LCC)  analyses, 
as  follows. 


SELECTION  OF  THE  BEST  CANDIDATE  EQUIPMENT/DESIGN 

Identify  all  candidate  equipments  - existing  and  to  be  developed  and  their  re- 
quirements for  operation  and  maintenance  during  the  deployment  phase  of  the  system 
life  cycle.  Eliminate  those  that  cannot  meet  minimum  requirements  for  the  new  mission. 
On  the  basis  of  total  life  cost  analyses,  the  equipment  having  the  lowest  total  life  cost  de- 
sign which  provides  the  minimum  essential  performance  should  be  selected  for  procurement. 
In  addition  to  total  life  cosiS,  quality,  deliver, , acquisition  costs,  and  political  factors  must 
be  considered  (although  total  life  cost  should  predominate)  so  that  the  greatest  project 
effectiveness  is  achieved. 


MILITARY  OFF-THE-SHELF  EQUIPMENT 

Determine  whether  there  are  any  off-the-shelf  military  (other  service  as  well  as 
Navy)  equipments  that  could  provide  the  required  characteristics  without  modification 
or  after  being  modified.  Obtain  the  existing  procurement  specifications  of  these  equip- 
ments and  one  of  the  units  of  each  to  review  their  characteristics.  Remember  that  even 
existing  military  equipment  may  require  new  interfaces  for  the  new  mission. 


FEASIBILITY  OF  MODIFYING  MILITARY  EQUIPMENT 

The  criteria  for  deciding  whether  or  not  modifications  to  military  off-the-shelf 
equipments  are  feasible  are: 

• Will  meet  new  requirements 

• Performance  will  not  he  degraded 

• Service  life  will  not  be  decreased 

• Availability  (reliability  and  maintainability)  will  not  be  decreased 

• Will  be  cost-effective 

COMMERCIAL  OFF-THE-SHELF  EQUIPMENT 

Determine  if  there  are  any  off-the-shelf  commercial  equipments  that  could  pro- 
vide the  required  characteristics,  without  modifications  or  after  modification.  See 
chapter  XI.  Screening  Techniques,  for  details.  I ven  without  mollification,  new  inter- 
laces may  be  required  for  the  military  mission. 


NEW  DESIGNS 


Since  the  costs  during  the  deployment  phase  (assumes  a quantity  greater  than  10) 
always  exceed  development  and  procurement  costs,  consider  new  design  concepts  that, 
compared  to  the  candidate  off-the-shelf  equipments,  are  aimed  at  less  expensive  operation, 
maintenance,  and  support.  Develop  or  expand  the  new  design(s)  to  the  extent  necessary 
to  provide  the  input  data  required  by  comparative  LCC  analysis  (see  chapter  IX.  Life- 
Cycle  Costing).  Be  sure  new  design  and  utilization  concepts  include  a well  planned  main- 
tenance concept  (see  MAINTAINABILITY  PROGRAM  in  Chapter  X)  and  the  use  of 
proved  technology  (no  forcing  the  state  of  the  art)  and  of  components  commonly  used  in 
industry. 

New  designs  are  often  preferred  in  order  to  obtain  the  advantage  of  in-house  con- 
trol which  allows  the  program  manager  to  influence  the  design.  Such  control  and  influence 
are  not  present  for  existing  equipments,  though  modifications  permit  some  opportunity 
for  reconfiguring  an  equipment  to  the  latest  desires  as  well  as  requirements. 

The  problems  associated  with  new  designs  involve  unknown  cost,  unknown  devel- 
opment time,  and  the  risk  that  the  new  development  will  not  perform  as  specified.  Any 
two  of  these  three  factors  may  be  defined  and  controlled,  but  the  third  one  will  always 
be  an  unknown.  Some  of  the  elements  that  comprise  the  development  costs  are:  labor, 
management,  documentation  (administrative,  procurement,  engineering,  and  support),  in- 
ventory (procurement,  storage,  and  control),  fabrication,  quality  assurance,  and  testing 
(performance,  environmental,  reliability,  maintainability,  safety,  etc).  There  is  also  a 
learning  curve,  wherein  the  first  units  always  cost  more  than  following  identical  units 
on  the  same  production  line. 

DEVELOPMENT  OE  MODIFICATIONS 

For  military  or  commercial  off-the-shelf  equipment  requiring  modification,  the 
modification(s)  will  require  developmental  effort.  The  selected  equipment  should  be 
developed  to  the  extent  that  a development  specification  (Type  B,  Mil  -S 10-490)  could 
be  prepared  in  sufficient  detail  to  effectively  describe  the  characteristics  which  must 
be  finalized  through  engineering  development  into  a production  model.  That  is,  the 
equipment  must  be  developed  sufficiently  at  this  point  so  that  the  requirements  tor  its 
engineering  development  can  be  specified  '•  II  enough  to  get  this  phase  ot  the  e 1 1 or t olt 
to  a good  start  and  end  in  a cost-effective  production  design  and  specification.  Check 
chapter  XVI.  Types  of  Contracts;  and  chapter  XV.  Warranty  Applications  Development 
of  modifications  should  include  considerations  similar  to  those  discussed  tor  new  designs 
in  the  next  paragraph. 


DEVELOPMENT  OE  NEW  DESIGNS 

In  new  designs,  all  equipment  characteristics  have  to  be  addressed  in  a develop- 
ment effort.  Refer  to  chapter  111.  Program  Planning.  The  extent  and  detail  ol  design 
delineation  required  at  this  point  for  a new  design  can  be  inferred  by  reference  to  Mil 
STD-490.  Appendixes  II  and  III,  which  define  the  content  requirements  ot  1 ype  B dev- 
elopment specifications. 


XIII  : 


LIFE-CYCLE  COST  ANALYSIS  FOR  DETAIL  DESIGN 


For  modification  or  new  designs,  urgent  consideration  should  be  given  to  the  use 
of  LCC  analysis,  commensurate  in  scope  and  expense  to  the  overall  project,  for  helping 
to  establish  the  design  details  that  must  be  delineated.  The  LCC  model  of  the  selected 
design  w1  :ch  was  used  for  the  comparative  analysis  can  be  amplified  to  be  of  value  at 
this  point.  It  then  can  be  updated  and  revised  as  it  is  used  to  help  determine  design  re- 
finements ! approved  engineering  changes  whic recur  during  engineering  development 
The  LCC  ar.  iivst  must  be  provided  the  necessary  additional  input  variables  relating  to 
equipment  characteristics  (see  chapter  IX.  Life-Cycle  Costing,  tor  use  in  this  LCC’  mod- 
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XIV.  CONSIDERATION  OF  STANDARDIZATION 


Standardization  is  a highly  controversial  subject.  The  essence  of  this  section  is 
that  DoD  requires  standardization  because  of  its  many  benefits.  Standardization  also 
has  many  disadvantages.  Fortunately,  there  are  various  degrees  of  standardization,  and 
it  is  wise  to  select  the  appropriate  degree  of  standardization  for  the  particular  program. 

The  Navy  has  committed  itself  strongly  to  a standardization  program  as  evidenced 
by  the  various  directives,  instructions,  standards,  and  specifications  on  the  subject.  J.  S. 
Gansler,  Assistant  Director  in  the  office  of  Defense  Research  and  Engineering,  states 
that  standardization  “is  a major  step  in  our  cost-reliability  attack"  and  that  “standard- 
ization can  and  must  be  achieved.”  In  essence  what  the  proponents  of  standardization 
are  saying  is  that  they  do  not  want  to  repeatedly  pay  for  developing  the  same  equip- 
ment. A highly  informative  review  of  this  subject  is  to  be  found  in  an  Electronics  “X” 
project  report  by  ARINC.  For  instance,  "The  reliabilities  achievable  for  SHP  (Standard 
Hardware  Program)  modules  are  no  greater  or  less  than  for  other  modules  of  similar 
complexity  and  technology  subjected  to  the  same  quality-control  provisions.”  Also, 
“Among  those  who  contributed  information  ...  it  was  the  consensus  that,  to  date, 
increased  costs  are  involved  in  the  development  of  systems  utilizing  SHP  concepts.” 

Those  within  industry  who  were  questioned  on  this  subject  for  the  TELCAM 
task  indicated  a dislike  of  the  idea  of  a requirement  to  use  standard  modules.  Telling 
the  manufacturer  how  to  design,  fabricate,  and  install  essentially  relieves  him  of  res- 
ponsibility for  the  result.  This  in  turn  destroys  incentive.  Actually,  industry  has  stan- 
dardized many  components,  procedures,  and  designs  through  their  joint  efforts  in  tech- 
nical organizations.  Acceptance  of  these  standards  is  voluntary,  but,  as  the  ARINC 
report  states.  “.  . . good  standards  do  not  have  to  be  ‘enforced.’  They  are  accepted 
because  they  make  economical  and  technical  sense  to  all  concerned.”  Further  standard- 
ization by  industry  is  accomplished  by  its  acceptance  of  rcuit  designs,  modules,  com- 
ponents, etc,  due  to  their  usage  history  within  the  industry.  This  standardization  may 
not  always  be  formalized  by  the  term  “standard,”  but  the  results  are  the  same.  These 
items  are  repeatedly  used  because  their  history  of  reliability,  cost-effectiveness,  replace- 
ability,  and  safety  contributes  to  a desirable  and  competitive  product. 

Many  of  those  interviewed  from  within  industry  felt  that  the' required  use  of 
SHP  modules  stifled  their  engineers’  creativity  and  meant  the  expenditure  of  time  and 
money  to  "design  in”  the  standardized  modules.  Industry  does  make  use  of  prevailing 
commercial  standard  components  and  parts  where  suitable  for  their  product:  but  where 
and  when  to  use  them  is  the  individual  decision  of  each  firm.  For  instance,  some  firms 
such  as  Motorola,  Scottsdale;  stock  nothing  but  high-reliability  parts  such  as  JAN-TX. 
Because  of  the  volume  of  parts  purchased  they  can  do  this  at  a price  equivalent  to  lower- 
reliability  part'  purchased  on  fragmented  buys.  These  parts  are  used  for  design  work  as 
well  as  production. 

Tlie  military  achieves  a form  of  standardization  by  repeatedly  procuring  identical 
equipments  over  a period  of  years.  This  results  in  technological  obsolescence  and  mediocre 
reliability. 

At  the  same  time,  other  similar  equipment  is  procured  for  slight  variations  in  mis- 
sion requirements  or  improved  equipment  capabilities  for  the  original  requirements,  or 
because  new  technology  surpasses  some  facet  of  the  original  equipment  technology.  Hi  is 
results  in  an  excessive  proliferation  of  alternative  equipments  and  defeats  any  benefits  ot 
repeatedly  procuring  identical  equipments. 

i 
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Where  only  minimal  organizational  maintenance  is  required,  a better  approach 
might  be  interface  standardization  at  the  level  of  the  black  box,  line  replaceable  unit,  or 
weapons  replaceable  (repairable)  assembly.  This  is  the  concept  now  used  in  the  Navy’s 
Standard  Missile. 

The  airline  industry  has  been  using  interface  standardization  for  years.  Combined 
with  functional  specifications,  they  call  it  “form,  fit,  function”  standardization.  The 
internal  configuration  is  free  to  evolve  as  technology  changes,  taking  advantage  of  new 
techniques,  devices,  and  materials.  Organizational  maintenance  consists  mainly  of  replac- 
ing black  boxes  for  repair  under  warranty  or  at  repair  depots. 

The  military  can  adopt  this  plan  and  standardize  the  functions,  interfaces,  com- 
ponents, and  workmanship.  Since  internal  circuits  and  configuration  are  not  specified 
or  standardized,  the  manufacturer  is  free  to  give  his  best  effort  in  the  internal  design. 
Competition  breeds  innovation  and  reduced  prices.  The  military  will  end  up  with  the 
functional  capability  specified,  the  optimum  technology  at  the  time  of  purchase,  and  the 
lowest  price.  Later  models  of  the  same  functional  item  will  be  different  internally,  but 
will  be  fully  interchangeable  with  the  initial  equipment.  The  need  for  modifications  to 
an  installation  to  accommodate  the  new  equipment  is  eliminated. 

Organizational  maintenance  will  not  extend  below  the  modular  level  or  perhaps  the 
black  box  level,  depending  on  mission  requirements  and  the  logistics  program  involved. 

The  maintenance  concept  will  include  a suitable  combination  of  throwaway  modules, 
warranties,  and  depot  maintenance.  See  chapter  X,  Integrated  Logistics  Support,  for  more 
details. 

General  worst-case  standards  have  been  developed  for  various  military  environments. 
Equipment  developed  to  withstand  such  environments  is  expensive.  Where  it  is  known  that 
the  actual  environment  for  an  equipment  will  be  less  severe  than  “standard.”  it  will  mean 
savings  to  modify  the  requirements  to  the  actual  environment.  This  is  permitted  by  MIL- 
E-16400 and  va.ious  environmental  standards  including  M1L-STD-810,  M1L-STD-167, 
M1L-STD-108,  and  M1L-STD-469. 

A well  organized  standardization  program  takes  advantage  of  standardization’s 
benefits  and  avoids  its  pitfalls.  To  accomplish  this,  the  project  manager  must  be  aware  of 
the  advantages  and  disadvantages. 


POTENTIAL  ADVANTAGES  OF  STANDARDIZATION 

1.  Standardization  reduces  life-cycle  cost. 

2.  It  makes  practical  a larger  design  budget  due  to  the  larger  number  of  units  for 
amortization  of  design  cost. 

1.  The  number  of  types  of  parts  to  be  purchased  and  stocked  is  reduced. 

I Standardization  reduces  the  unit  cost  of  necessary  repair  parts  by  requiring 
larger  quantities  of  relatively  fewer  types. 

S It  increases  the  quantity  of  the  same  item,  which  reduces  its  unit  cost  through 
economies  of  scale,  mechanized  processes,  and  longer  production  runs  of  fewer 
designs. 

I irger-qu.intity  productions  make  practical  the  use  ol  LSI  integrated  circuits. 
|li  number  of  odd  or  unusual  items  is  reduced. 
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8.  Production  cost  is  reduced  in  terms  of  tooling  due  to  higher  usage  of  fewer 
setups. 

9.  Less  types  of  test  equipment  are  required. 

10.  Simplified  test  procedures  and  fault  isolation  become  possible.  Automated 
test  equipment  becomes  more  practical  to  design  anil  produce. 

I 1.  Training  requirements  and  skill  levels  required  for  maintenance  are  reduced. 

12.  Standardization  increases  maintainability  and  minimizes  equipment  downtime 
due  to  common  spares  and  off-line  repair  or  throwaway. 

13.  The  number  of  different  documents  is  reduced. 

14.  Predictability  is  increased  for  reliability,  maintainability,  safety,  costs,  etc. 

15.  Reliability  is  increased  through  evolutionary  redesign. 

16.  Availability  is  increased  due  to  increased  reliability  and  maintainability. 

17.  Universal  packaging  becomes  possible  through  standardization. 

18.  Uniformity  in  size,  shape,  and/or  connectors  simplifies  storage  and  testing 
requirements. 

19.  Standardization  permits  parts  or  modules  to  be  interchanged  between  or 
within  systems  and  equipments. 

20.  It  provides  modular  "building  blocks”  for  electronic  and  mechanical  design 
(which  speeds  design  and  reduces  development  time). 

21.  Time  and  expense  are  saved  in  determining  optimum  enclosure  size,  module 
size,  circuit  board  size,  etc. 

22.  Standardization  reduces  redundant  design  efforts,  reinventing  the  wheel,  or 
redeveloping  existing  equipment. 

23.  It  permits  universal  use  of  carefully  designed  solutions  to  problems  such  as 
cooling,  electromagnetic  interference,  shock,  and  vibration. 

24.  The  number  of  similar  equipments  is  reduced  along  with  their  development 
time,  costs,  and  documentation. 

25.  Multiple  use  or  broader  application  of  an  item  becomes  possible. 

26.  The  cost  of  adding  a nonstandard  item  to  the  stock  system  is  eliminated. 

27.  Instructive  guidelines  are  provided  for  the  uninitiated,  which  allows  novices 
to  compete  for  the  development  and  production  of  military  electronic  equip- 
ment. 

There  are  always  some  things  we  would  like  to  see  done,  but  the  question  is  how  much 
we  are  willing  to  pay  for  it  vs  how  much  we  will  actually  benefit  from  it.  Military  stand- 
ards and  specifications  tend  to  be  excessively  long  and  complex,  slow  to  follow  the  rapid 
advances  in  technology,  and  difficult  and  expensive  to  implement.  They  often  fail  to 
produce  the  intended  results.  Numerous  military  standardization  programs  have  been  tried. 
Most  have  died  through  lack  of  use.  The  causes  for  such  lack  of  use  are  among  the  lol low- 
ing disadvantages  of  standardization. 
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TYPICAL  DISADVANTAGES  OF  STANDARDIZATION 

Standardization  “benefits”  are  often  ideals  that  cannot  be  realized. 

Standards  are  usually  prepared  by  designers  of  standards  rather  than  designers 
of  equipmen  . 

High  initial  costs  are  inherent  in  standardization  programs. 

Components  which  are  larger,  more  expensive,  or  unnecessary  except  for 
standardization  purposes  add  to  equipment  and  repair  costs. 

Repair  by  replacement  of  expensive  subassemblies  is  not  cost-effective  in 
cases  where  it  is  obvious  that  an  individual  component  has  failed.  Standard- 
ization at  the  modular  level  can  lead  to  difficulty  in  locating  what  would 
otherwise  be  a simple,  low-cost  repair  of  an  obvious  failure. 

Standardization  limits  design  flexibility  and  alternatives  (which  increases  cost). 
Standardization  cannot  satisfy  all  requirements  (can't  please  everyone). 

Standardization  tends  to  have  a narrow  view  and  does  not  always  consider 
all  parameters  as  variables,  but  rather  fixes  certain  ones  as  constants.  This 
results  in  unreasonable  constraints  on  the  designer  and  possibly  unreliable 
equipment  in  service. 

It  usually  involves  less  than  optimum  electrical  and  mechanical  design  para- 
meters. “Optimum”  for  one  circuit  system,  etc.  is  not  optimum  for  the  next. 

Generally  inefficient  systems  result  from  standardization.  Problems  include 
excessive  weight,  excessive  volume,  excessive  heat,  and  excessive  complexity. 
Standards  designed  for  expected  worst-case  conditions  produce  inherently 
inefficient  results,  and  cost  more  than  optimum  items. 

Equipment  standardized  internally  has  low  volumetric  efficiency  due  to  addi- 
tional required  connectors  and  hardware,  and  larger  than  optimum  (but 
standard)  connectors,  hardware,  and  components,  all  assembled  in  a larger 
than  optimum  (but  standard)  module,  subassembly,  case,  cabinet,  etc. 

Standardization  generally  conflicts  with  "best  commercial  practices.” 

Equipment  designed  to  a multitude  of  standards  to  fit  a variety  of  needs 
generally  suits  none  of  the  needs  well.  Due  to  extensive  tradeoffs  and  com- 
promises, the  result  may  be  unsuitable  for  any  of  the  intended  applications. 

Standard  modules,  circuits,  etc,  are  frequently  peculiar  to  the  one  system  for 
which  they  were  originally  designed,  and  cannot  be  designed  into  another 
system  without  causing  deficiencies,  inefficiency,  or  unnecessary  complexity. 

Items  designed  and  built  to  general  standards  have  trouble  adapting  to 
special  requirements. 

There  can  be  otherwise  unnecessary  and  cumbersome  interface  restrictions 
required  for  the  sake  of  standardization;  eg,  connectors,  voltage  levels,  and 
module  sizes. 

An  increase  in  the  number  of  connections  leads  to  reduced  reliability. 

Increased  conductor  lengths  may  cause  cross  talk  or  noise,  or  other  l-'MI 
problems. 
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ll).  Long  lead  times  (often  due  to  low  usage)  are  common  for  standard  items. 

20.  Commercial  managers  may  be  unwilling  to  change  to  military  standards  from 
their  in-house  standards  for  psychological,  economic,  or  sound  technical 
reasons. 

21.  Standardization  can  add  to  the  complexity  of  fault  isolation. 

22.  When  a defective  subassembly  is  isolated,  a modular  replacement  philosophy 
may  preclude  module  repair  due  to  lack  of  detailed  internal  information  or 
lack  of  repair  parts. 

23.  Standardization  requires  that  adequate  documentation  regarding  the  standard 
be  made  available  to  users. 

24.  It  tends  to  be  arbitrary  for  the  sake  of  documentation. 

25.  Sometimes  "the  tail  wags  the  dog”:  standardization  controls  and  directs  the 
program  rather  than  being  a tool  of  the  project  manager. 

26.  Standardization  is  sometimes  more  politically  than  economically  or  techni- 
cally motivated. 

27.  Standards  become  obsolete  even  if  valid  when  written  (often  obsolete  before 
published).  This  limits  improvements  in  performance,  cost,  size,  weight,  or 
reliability.  A good  (rather  than  arbitrary)  standardization  program  requires 
time  to  develop.  Unfortunately,  this  sometimes  leads  to  a “new  standard” 
which  becomes  obsolete  before  it  is  finalized. 

28.  Standardization  programs  generally  do  not  last.  Needs,  missions,  and  state 
of  the  art  are  all  constantly  changing.  Standardization  is  an  attempt  to  hold 
on  to  the  present  and  assumes  that  the  particular  standard  will  remain  useful 
for  an  indefinite  period  of  time. 

In  theory,  the  advantages  of  standardization  clearly  outweigh  the  disadvantages. 

In  the  real  world,  however,  “standardization”  is  often  no  more  than  a management 
technique  to  maintain  control  over  an  otherwise  constant  stream  of  changes.  The  changes 
may  be  improvements,  but  if  designers  continue  to  improve,  nothing  ever  gets  built. 

Excessive  or  mismanaged  standardization  can  be  bad  and  can  become  very  expen- 
sive and  yield  obsolete  systems  with  excessive  size,  weight,  etc. 

There  is  always  some  natural  standardization  in  any  project.  The  questions  arc 
how  much  and  whether  it  should  be  enforced.  If  enforced,  how  much  enforcement  and 
by  what  means.  Allow  flexibility  in  the  application  of  specifications  and  standards  to 
increase  cost-effectiveness.  Encourage  standardization  as  a means  of  life-cycle  cost  reduc- 
tion, but  remember  that  excessive  standardization  can  lead  to  excessive  complexity  and 
excessive  cost. 

In  order  to  develop  standards,  many  variables  that  are  extreme  for  one  purpose 
must  be  made  extreme  for  all  uses  (which  drives  up  the  cost  of  the  system),  or  else  must 
be  made  less  extreme  to  reach  a compromise  between  the  various  requirements  (which 
may  make  the  system  unsuitable  for  any  purpose).  There  is  a third  alternative:  to  develop 
a standard  with  specified  options  or  defined  classes  of  use  to  meet  the  independent  needs 
of  different  users. 

Another  use  of  standards  is  to  put  on  paper  constantly  changing  interface  require 
ments.  This  ensures  that  two  or  more  groups  designing  portions  of  a system  will  have  a 
common  meeting  ground.  Only  interface  standards  (form.  fit.  and  function)  are  required 
for  this. 
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DEFENSE  STANDARDIZATION  PROGRAM  (DSP) 


The  Navy’s  role  in  the  Defense  Standardization  Program  is  outlined  in  SECNAV- 
INST  4120.3B.  An  enclosure  to  the  instruction  is  DoD  Directive  4120.3,  which  defines 
staiula  li/ation  as  the  adoption  and  use  (by  consensus  or  decision)  of  engineering  criteria 
to  achieve  the  following  objectives: 

1.  To  improve  the  operational  readiness  of  the  military  services  by  increasing 
efficiency  of  design,  development,  material  acquisition,  and  logistics  support. 

2.  To  conserve  money,  manpower,  time,  facilities,  and  natural  resources. 

3.  To  minimize  the  variety  of  items,  processes,  and  practices  which  are  associ- 
ated with  design,  development,  production,  and  logistics  support  of  equip- 
ments and  supplies. 

4.  To  enhance  interchangeability,  reliability,  and  maintainability  of  military 
equipments  and  supplies. 

DSP  PLANS  FOR  ACHIEVEMENT 

The  Defense  Standardization  Program  seeks  to  achieve  these  objectives  through. 

1.  Management  and  engineering  actions  required  to  establish  and  effectively  im- 
plement standardization  agreements  anil  decisions. 

2.  Establishing  and  maintaining  uniform  and  technically  adequate  records  of  the 
engineering  definition  of  equipments  and  supplies. 

3.  Promoting,  in  support  of  procurement,  maintenance,  supply,  and  future  design, 
the  reuse  of  records  of  engineering  definitions,  the  engineering  criteria  therein, 
and  previously  developed  or  acquired  material  represented  by  these  records. 

4.  Prescribing,  for  specifications,  standards,  drawings,  and  other  standardization- 
association  documental  on.  (a)  format,  (b)  procedure  for  effective  coordina- 
tion, (c)  quality  of  documentation,  and  (d)  procedures  for  collating  and  dis- 
seminating this  information. 


POLICIES  OF  THE  DSP 

1.  Military  Operational  Requirements.  Wherever  feasible,  military  operational 
requirements  for  material  shall  be  satisfied  through  the  use  ol  existing  military 
designs  or  commercial  products.  If  a military  need  can  be  satisfied  only 
through  new  development,  the  new  development  authorized  shall  encompass, 
to  the  greatest  extent  practicable,  all  equivalent  needs  ot  the  Military  Depart- 
ments and  Defense  Agencies. 

2.  Exploratory  Development  and  Advanced  Development.  The  categories  ot 
Exploratory  Development  and  Advanced  Development  represent  scientific, 
experimental,  and  early  development  efforts  aimed  at  innovation  and  evalu- 
ation of  feasibility.  The  use  of  existing  standard  items  and  engineering  prac- 
tices is  advocated  in  the  interests  of  economy,  where  these  satislv  the  needs 


of  such  program  efforts.  However,  use  ot  standards  shall  be  secondary  to 
the  prime  objective  of  these  development  categories;  eg,  proot  ot  a concept. 

3.  Engineering  Development  and  Operational  System  Development.  In  the  cate- 
gories of  Engineering  Development  and  Operational  System  Development  where 
the  systems  and  equipment  are  engineered  for  eventual  Service  use,  a maximum 
degree  of  standardization  shall  be  achieved  without  causing  unacceptable 
compromise  of  performance,  reliability,  timely  availability,  or  cost  of  systems 
and  without  preventing  the  applications  ol  the  most  advanced  proved  tech- 
niques or  hardware. 

4.  Procurement.  Techniques  that  provide  opportunities  to  achieve  standardiza- 
tion objectives  during  procurement  include  (a)  the  specification  ol  complete 
design  requirements,  (b)  multiyear  procurement,  and  (c)  negotiation  author- 
ized by  Paragraph  3-213  of  Armed  Services  Procurement  Regulation  to 
achieve  standardization  of  technical  equipment. 

5.  Supply.  The  variety  of  types,  kinds,  and  sizes  of  items  of  supply  shall  be 
reduced  to  the  minimum  consistent  with  effective  support  of  military 
operations. 

6.  Standardization  Planning.  Standardization  efforts  will  be  planned  and  managed 
with  a view  to  establishing  (a)  timely  design  standards  that  reflect  current 
technology;  and  (b)  standards,  specifications,  and  other  documentation  that 
offer  the  greatest  advantages  for  cost  reduction,  for  item  reduction,  for  com- 
petitive procurement,  for  simplification  of  maintenance  and  logistics  support, 
for  increased  reliability,  or  for  increased  design  efficiency. 

7.  Management  of  Engineering  Information. 

a.  Organization  of  Engineering  Information.  Engineering  information,  ob- 
tained from  design  and  development  and  of  the  type  reasonably  expected 
to  be  necessary  for  future  reuse,  shall  be  organized  in  standard  format 

to  promote  repetitive  use  in  support  of  procurement,  production,  main- 
tenance. supply,  and  new  design. 

b.  Specifications.  When  required  for  design  support,  configuration  identi- 
fication, or  procurement,  an  adequate  engineering  definition  shall  be 
established  for  material  and  practices  resulting  from  new  development 
(engineering  and  operational  systems  development),  and  for  items 
authorized  for  production  or  supply  support.  The  preferred  format  is 
the  Federal  or  Military  specification. 

c.  Industry  Documents.  Specifications  and  standards  of  nationally  recog- 
nized industry  organizations  and  technical  societies  shall  be  used  in  the 
development  and  design  of  material,  and  in  the  preparation  ol  Military 
or  Federal  specifications  or  standards  to  the  maximum  extent  practica- 
ble. Duplication  in  the  Military  series  of  industry  standards  is  to  be 
avoided. 

K.  Industry  Coordination.  The  procedures  for  Department  of  Defense  Standard- 
ization will  provide  for  industry  participation  to  an  appropriate  degree  and 
will  avoid  duplication  of  effort  Consideration  will  be  given  to  the  contractual 
responsibilities  of  contractors  in  the  design-development-production  cycle  to 
prepare  required  documentation  and  also  to  the  needs  ol  contractors  in  the 
use  of  documentation  produced  under  this  program. 


RESPONSIBILITIES  ASSIGNED  BY  THE  DSP 

The  Defense  Standardization  Program  assigns  responsibilities  as  follows. 

1.  Engineering  Determinations.  Design,  development,  and  engineering  activities 
are  responsible  for  the  engineering  determinations  recorded  in  specifications, 
standards,  drawings,  and  criteria  for  interchangeability  and  substitutability 
(when  the  criteria  are  not  contained  in  military  design  standards). 

2.  Record  Preparation.  Design,  development,  and  engineering  activities  are  re- 
sponsible for  the  timely  preparation  of  records  of  new  development  in  au- 
thorized format  for  the  support  of  configuration  management,  production, 
procurement,  and  follow-on  logistics  support.  These  activities  also  are  respon- 
sible for  recording  and  approving  any  justified  use  of  nonstandard  parts,  com- 
ponents, and  materials. 

3.  Use  in  New  Design.  The  design  and  development  activities  have  responsibility 
for  the  use  in  new  design  of  (a)  applicable  standards,  (b)  suitable  items  avail- 
able in  supply,  and  (c)  suitable  commercial  items,  before  any  new  develop- 
ment is  authorized  to  meet  equipment  or  systems  objectives. 

4.  Users.  Users  of  engineering  information  are  responsible  for  the  formulation 
of  programs  (eg,  item  reduction,  item  entry  control,  configuration  manage- 
ment, and  design  selection  discipline)  to  achieve  maximum  benefit  from  use 
of  standards  and  other  standardization  documents. 

5.  Logistics  Support.  Inventory  Managers  are  responsible  for  programs  (a)  to 
screen  items  seeking  entry  into  supply  through  provisioning,  (b)  to  prevent 
identical  items  from  entering  with  differing  identifications,  and  (c)  to  sub- 
stitute. for  the  new  item,  an  interchangeable  or  substitutable  item  already  in 
the  supply  system,  or  an  available  item  covered  by  a military  standard  or 
specification.  Supply  management  is  responsible  for  limiting  the  number  of 
different  makes  and  models  of  equivalent  equipment  in  any  geographical  area. 


STANDARDIZATION  OF  COMPONENTS/EQUIPMENTS  (C/E) 

One  of  the  Navy's  standardization  efforts  is  outlined  in  NAVMATINST  41 20. 97 A. 
In  this  instruction,  Components/Equipments  (C/E)  is  defined  as  repairable  items  which  re- 
quire repair  part  support. 

The  Navy  Component/Equipment  Program  was  established  to  curb  the  proliferation 
of  components/equipments  being  introduced  into  the  fleet.  Several  logistics  studies  had  re- 
ported the  existence  of  many  nonstandard  and  low-population  equipments  requiring  dif- 
fering repair  parts  which  were  difficult  to  support.  Also,  such  nonstandardization  has  con- 
tributed directly  to  proliferation  of  allowance  parts  lists,  technical  manuals,  configuration 
management,  training,  and  other  logistic  requirements. 


POLICIES  OF  THE  C/E  PROGRAM 

The  policy  of  the  C/E  Program  applies  to  all  elements  of  the  Naval  Material  Com- 
mand m all  phases  of  system  development,  acquisition,  and  maintenance.  It  includes  all 
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systems,  subsystems,  components,  and  equipments.  Stated  in  the  broadest  terms,  the 
policy  is  to: 

1.  Include  hardware  standardization  requirements  in -concept  formulation,  con- 
tract definition,  procurement,  production,  maintenance,  conversion,  modern- 
ization, and  alteration. 

2.  Standardize  designs  - with  intersystem  and  intrasystem  standardization  of 
C/E  and  parts. 

3.  Reuse  (in  new  design)  existing,  suitable  C/E  already  supported  in  depth  by 
the  military  system. 

4.  Preclude  use  of  limited-application  and  poor-performance  C/E. 

5.  Exercise  configuration  control  to  maintain  standardization. 

6.  Use  procurement  techniques  to  restrain  proliferation. 

7.  Effect  item  entry  control  in  design  selection  and  provisioning  phases  of  mate- 
rial acquisition. 


C/E  PLAN  OF  OPERATION 

1.  The  plan  for  increasing  C/E  standardization  is  to: 

a.  Promulgate  and  implement  the  policy  set  forth  above. 

b.  Give  visibility  to  ongoing  standardization  effort  by  indexing  it. 

c.  Promote  the  use  of  standardized  and/or  existing  C/E. 

d.  Backfit  standardization  into  existing  systems. 

2.  The  attainment  of  an  optimum  degree  of  standardization  by  curbing  C/E  mak 
and  model  multiplication  and  resultant  spare  parts  proliferation  must  be  with- 
in the  bounds  of  ASPR  (Armed  Services  Procurement  Regulation)  and  any 
governing  requirements  of  the  Defense  Standardization  Program.  Standardiza- 
tion cannot  be  mandated  arbitrarily  but  must  result  from  thoroughly  consi- 
dered tradeoffs,  generally  on  the  basis  of  cost  versus  effectiveness. 

Each  Project  Manager  shall  maintain  his  own  internal  plan  for  standardization 
of  C/E  under  his  technical  cognizance.  (NOTE’:  Project  Managers  will  not 
be  required  to  develop  separate  standardization  plans  if  their  PMPs  (Project 
Master  Plans)  specifically  address  requirements  for  standardization  of  C l ) 
Each  plan  shall  stipulate  the  development  of  indices  to  reflect  the  current  sit- 
uation of  standardization  in  major  commodity  areas  (eg.  Hull  Mechanical 
Electrical,  Electronics  Communications  Equipment.  Electrical/Electronics  I e\l 
Equipment,  Aviation  Ground  Support  Equipment.  Avionics,  Aviation  Ord- 
nance, Conventional  Ordnance.  Guided  Missiles)  against  which  to  plan  and 
measure  future  accomplishment.  The  plan  will  provide  for 

a.  Implementing  the  standardization  policy  in: 

( I ) Concept  formulation,  contract  definition,  and  other  phases  of  mate- 
rial acquisition  planning,  including  I’MPs  (Project  Master  Plans!,  Re- 
search and  Development  Planning,  and  API’s  (Advanced  Procurcmen 
Plans). 
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(2)  Procurement  of  C/E  with  use  of  standardization  requirements  clauses, 
where  warranted;  life-cycle  costing:  and  central  procurement  of  GFE/ 
CFE  to  effect  consolidated  buys  of  standardized  (identical  in  make  and 
model)  C/E.  The  Standardization  Exception  (ASPR  3.213),  covering 
technical  equipment  requiring  standardization  and  interchangeability  01 
parts,  will  be  utilized  wherever  the  stipulations  contained  therein  can 
be  met  (for  the  purposes  of  application  of  the  exception  a “standard 
item”  is  any  item  already  in  use  in  the  Navy  (eg,  an  item  identified  by 
a FSN  (Federal  Stock  Number)  and/or  an  APL/CID  (Allowance  Parts 
List/Component  Identification)  number;  excluded  are  items  designated 
nonstandard  in  the  Federal  cataloging  records).  It  will  be  specified  that 
all  Systems/Subsystems/components/equipments/instrumentation  re- 
quiring repair  part  support  are  to  be  of  identical  make  and  model  (having 
identical  internal  parts)  within  the  block  of  hardware  being  bought  under 
any  individual  contract. 

b.  Developing,  whenever  practicable,  standardized  design  for  C/E. 

e.  Selecting  C/E  for  new  systems  from  those  equipments  which  are  pres- 
ently supported  (operationally  and  logistically)  in  depth. 

d.  Utilizing  subsystems  of  one  system  in  other  systems  requiring  similar 
design  and  performance. 

e.  Assuring  that  a minimum  variety  of  C/1-  is  used  in  system  design  by  in- 
corporating standard  C/E  with  better  performance,  or  other  values, 
wherever  significant  logistics  benefits  will  accrue  and  can  be  measured 
on  a life-cycle  cost  savings  basis. 


STANDARD  HARDWARE  PROGRAM  (SHP) 


M IL-F- 1 6400G.  paragraph  3. 3. 2. 3,  requires  the  consideration  of  the  packaging 
techniques  described  in  NAVSIIIPS  05 1 8-043-2500,  which  includes  the  SHP.  A further 
requirement  is  modular  construction  conforming  to  MIL-STD-1 378.  again  SHP. 

NAVMATINST  4 1 2 0 . 1 0 2 A states  that  the  requirements  of  the  Standard  Hard- 
ware Program  apply  to  the  initial  development  or  redesign  of  ship  and  shore  electronic 
equipment  and  systems.  The  program  is  optional  for  other  types  of  equipment  such  as 
avionic  equipment,  satellites,  and  portable  equipment. 

All  PMs  will  participate  in  the  SHP  to  the  extent  that  SHP  modules  will  be  used 
m new  systems  where  technically  and  economically  feasible.  The  features  of  the  SHP 
that  concern  maintaining  and  controlling  the  use  of  the  standards  will  also  be  observed. 
The  instruction  requires  that  PMs  analyze  and  assess  the  feasibility  of  using  SHP  modules: 
and  that,  where  the  analysis  indicates  the  SHP  modules  to  be  technically  and  economi- 
cally suitable,  use  of  the  modules  shall  be  specified  in  the  development  of  the  system. 
Once  this  decision  has  been  made,  the  SHP  procedural  rules  established  by  NAVI  1 I \ 
to  maintain  integrity  of  the  standards  shall  be  observed.  The  requirements  do  not.  how- 
ever. relieve  responsibility,  nor  abrogate  the  authority,  of  PMs  as  the  technical  and  man- 
agement agents  over  their  programs  in  accordance  with  established  directives  As  pre- 
viously stated,  the  project  manager  has  the  option  of  using,  not  using,  or  discontinuing 
use  of  SHP  modules  on  the  basis  of  legitimate  and  demonstrable  technical  or  economic 
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factors.  Increased  participation  in  the  SHP  is  desired,  and  will  be  of  mutual  benefit  to 
all  concerned,  but  established  authority  of  PMs  will  continue  to  be  recognized. 

PMs  who  require  testing  services  for  modules  under  a specific  project  at  the  SHP 
Quality  Assurance  Activity  (NAI),  Crane)  will  be  responsible  for  the  cost  of  the  testing 
and  related  overhead. 

PMs  will  deal  directly  with  the  Quality  Assurance  Activity  through  normal  chan- 
nels. Services  performed  by  the  Design  Review  Activity  (NAF1,  Indianapolis)  that  are 
related  to  the  overall  SHP  standardization  program  will  be  funded  by  NAVFLHX  and 
made  available  to  other  SYSCOM  and  PMs.  NAVELFX  will  provide  assistance  to  other 
SYSCOM  and  PMs  in  applying  SHP. 


RESPONSIBILITIES  UNDER  SHP 

All  PMs  will: 

1.  Review  all  planned  projects  within  the  scope  outlined  above  for  technical 
and  economic  applicability  of  the  SHP. 

2.  Where  review  indicates  feasibility  for  application  of  the  SHP.  require  its  use 
in  contract  specifications  by  citation  of  MIL-STD-1378. 

3.  For  those  programs  whose  combined  total  estimated  cost  for  R&D  and  ini- 
tial production  exceeds  $1  million,  provide  notification  to  NAVELFX.  in 
format  of  RCS  NAVMAT  4120-10,  that  the  use  of  SHP  has  been  considered 
if  not  used,  give  the  details  thereof. 

4.  Provide  for  the  production  sampling  tests  at  the  SHP  Quality  Assurance  Ac- 
tivity on  existing  standard  modules  that  will  be  used  in  the  project. 

5.  Provide  for  qualification  tests  at  the  SHP  Quality  Assurance  Activity  on  new 
standard  modules  that  will  be  needed  in  this  project. 

6.  Provide  for  contractor  development  and  delivery  of  SHP  documentation  for 
new  standard  modules  resulting  from  the  project. 

7.  Recommend  to  NAVELEX  any  techniques  found  effective  in  specific  proj- 
ects which  could  or  should  be  applied  Navy-wide. 

8.  Provide  points-of-contact,  technical  panel  representation,  and  supporting  in- 
formation, upon  request,  to  assist  NAVELFX  in  SHP  management. 


STANDARDIZATION  REFERENCES 

1.  Naval  Electronic  Systems  Command 

MIL-M-28787.  Modules,  Electronic.  Standard  Hardware  Program.  General  Speci- 
fication for  (supersedes  WS61  16,  in  part).  14  Mar  73 

2.  Naval  Electronic  Systems  Command 

M I L-S  I D- 1 3 78A . Requirements  tor  Employing  Standard  Hardware  Program  Mod- 
ules. 14  Mar  73 

3.  Naval  Electronic  Systems  Command 

MIl.-STD-l  389,  Design  Requirements  for  Standard  Hardware  Program  Electronic 
Modules.  14  Mar  73 
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4.  Naval  Sea  Systems  Command  NAVSH1PS 

0518-093-2500  (formerly  NAVMAT  P-3940),  Navy  System  Design  Guidelines 

Manual,  May  67 
— 

5.  Naval  Electronic  Systems  Command 

NAVELEX  0101-053B,  Standard  Hardware  Program  Module  Index.  Oct  72* 

6.  Naval  Electronic  Systems  Command 

NAVELEX  0101-051,  Program  Managers  Guide  for  the  Standard  Hardware  Pro- 
gram* 

7.  Naval  Ordnance  Systems  Command  NAVORD 

OD  30355.  Standard  Hardware  Program  Data  Handbook* 

8.  Naval  Ordnance  Systems  Command  NAVORD 

OD  36739,  Standard  Hardware  Program  Drawings,  Preparation  of* 

9.  Department  of  Defense  DoD  Directive  41 20.3 M 

10.  Secretary  of  the  Navy  SF.CNAV1NST  4120. 3B, 

“Defense  Standardization  Program,”  25  Jan  68 

11.  Naval  Material  Command  NAVMAT1NST  4120.97A. 

“Standardization  of  Components/Equipments  (C/E)  Required  for  E'leet  or  Ashore 
Support,”  1 7 Feb  71 

12.  Naval  Material  Command  NAVMAT1NST  4120.102A 
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GENERAL 

The  use  of  warranties  in  contracts  is  intended  to  assure  the  buyer  of  — and  make 
the  seller  liable  for  a specified  performance  of  the  system/equipment  being  procured. 
For  government-purchased  electronic  equipment  such  assurance  has  become  a matter  of 
great  concern  because  supporting  the  unreliable  performance  of  many  of  these  items  has 
become  increasingly  more  costly.  There  is  an  urgent  need  to  reduce  support  costs  to 
levels  that  are  consistent  with  today’s  stringent  funding  limitations,  and  DoD  is  taking  a 
new  look  at  acquisition  strategy  as  one  way  to  solve  this  cost  problem.  The  new  look 
is  being  directed  in  part  at  the  use  of  types  of  warranties  that  are  more  extensive  than 
the  standard  warranties  for  which  ASPR  has  clauses  and  that  attack  the  support-cost 
problem  directly. 

A significant  portion  of  the  increasing  cost  of  maintenance  and  support  is  due 
to  poor  reliability.  It  is  axiomatic  that  the  greater  the  time  between  failures  (or  the 
fewer  the  failures),  the  less  the  support  cost  will  be  both  for  per-unit  cost  of  repair  and 
for  cost  of  spares.  Thus,  effective  effort  to  improve  reliability  of  an  equipment  will 
yield  reduced  support  costs,  not  to  mention  the  significant  benefit  of  improved  avail- 
ability. 

The  “new”  warranty  concept  the  government  is  considering  is  aimed  at  this  re- 
liability and  maintainability  improvement.  It  adopts  a warranty  practice  the  airline 
industry  has  been  successfully  using  for  some  time  to  improve  reliability  and  lessen 
maintenance. 

Conventional  government  procurements  have  the  contractor's  liability  ending 
with  delivery  and  government  acceptance,  with  reliability  and  maintainability  (R&M) 
accomplished  by  specifying  numerical  requirements  and  applicable  military-standard  pro- 
grams during  design,  development,  and  production.  The  efforts  have  been  partly  suc- 
cessful, but  have  not  produced  the  desired  and  needed  results.  Too  often  the  govern- 
ment has  been  unable  to  exactly  define  reliability  requirements  and  measure  reliability 
attainments.  This  has  made  it  difficult  to  levy  stiff  penalties  on  contractors  for  failure 
to  achieve  required  full  reliability,  and  has  resulted  in  deviations  and  waivers  often  being 
allowed  by  the  government  and  in  very  little  use  of  available  reliability-incentive  clauses 
in  procurement  contracts. 

Working  to  aggravate  this  problem  is  the  contractor’s  inclination  to  maximize 
his  profits.  As  he  tries  to  improve  profits,  reliability  suffers  because  high  reliability 
costs  him  more  to  produce.  Regardless  of  the  intent  of  reliability  specification,  demon- 
stration, and  test  requirements,  the  contractor  has  the  final  say  about  the  reliability 
level  of  the  item  being  produced  simply  because  he  is  the  one  actually  doing  the  job. 
And  if  he  is  motivated  more  by  profit  than  by  product  excellence,  the  delivered  product 
more  often  than  not  will  be  barely  acceptable  and  then  on  the  basis  of  dev  iations  am' 
waivers. 

Enter  the  “Reliability  Improvement  Warranty"  (RIW).  The  RIW  is  one  of  the 
several  types  of  warranties  intended  for  improving  R&M  which  are  discussed  in  this  sec- 
tion. and  one  to  which  DoD  is  giving  particular  attention  at  the  time  of  this  writing. 

The  RIW  works  to  take  advantage  of  contractor  profit  motivation  rather  than  working 
at  cross  purposes  like  the  conventional  procurement  contract.  If  the  contractor  is  made 
responsible  for  long-term  reliability  after  delivery  and  during  operation,  and  for  a fixed 
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fee  that  allows  him  to  increase  profits  in  proportion  to  the  level  of  reliability  and  the 
ease  of  maintenance  that  are  achieved,  the  contractor  is  inclined  to  burn  the  midnight 
oil  on  his  R&M  programs.  The  RIW  becomes  a contracting  technique  by  which  the 
government  derives  the  benefits  of  improved  reliability  and  maintainability  for  each  in- 
dividual dollar  the  contractor  earns.  Other  warranties  of  this  nature  approach  the  R&M 
problem  a little  differently,  as  will  be  noted. 

Before  discussing  the  individual  warranties  designed  to  lessen  the  cost  of  R&M. 
a few  words  are  in  order  about  the  general  requirements  of  ASPR  for  all  warranties 
including  the  R&M  ones. 

Two  basic  types  of  warranties  exist  based  on  the  Uniform  Commercial  Code  (UCC) 
to  provide  buyer  protection.  No  federal  law  conflicts  with  the  UCC.  and  the  government 
has  adopted  it  for  government  contracting.  The  first  type  of  warranty  is  an  implied  one. 

In  government  procurements  the  existence  of  an  implied  warranty  depends  on  the  type  ol 
contract;  it  does  not  apply  to  cost  reimbursement  contracts.  The  second  type  of  warranty 
is  an  expressed  one.  a clause  being  included  in  the  contract  to  define  the  conditions  and 
provisions  of  the  warranty.  Reliability  improvement  and  other  maintenance  and  repair 
oriented  warranties  are  express  warranties. 

Defense  Procurement  Circular  74-2,  issued  4 October  1 0 74 . revised  the  Warrant) 
Section  of  ASPR  and  added  new  clauses  for  conventional-type  warranties.  The  new 
clauses  do  not  differ  significantly  from  the  former  except  for  a major  cluing  - which  states. 
"All  implied  warranties  of  merchantability  and  fitness  for  a particular  purpose  are  here- 
by excluded  from  any  obligation  contained  in  this  contract." 

Two  purposes  are  given  for  warranties:  to  delineate  rights  and  obligations  re- 
garding defective  items,  and  to  foster  quality  performance.  ASPR  policy  is  that  a war- 
ranty clause  shall  be  used  when  it  is  found  to  be  in  the  best  interest  of  the  government 

New  provisions  involve  the  pricing  aspects  for  fixed-price  incentive  warranty  pro- 
visions and  differ  from  the  previous  ASPR.  The  new  section  says  the  estimated  costs  ol 
warranty  coverage  should  not  normally  be  considered  in  the  incentive  target  price,  and 
all  costs  should  be  considered  in  establishing  a ceiling  price.  All  warranty  costs  incurred 
are  to  be  considered  then  in  the  total  final  price.  And  after  establishment  of  the  total 
final  price,  the  contractor  is  to  bear  all  warranty  costs. 


RELI  ABII.ITY  IMPROVEMENT  WARRANTY  (RIW) 

DoD  has  requested  the  Armed  Services  to  initiate  RIWs  on  a trial  basis,  to  help 
determine  the  scope  and  benefits  that  these  warranties  may  have.  In  a memorandum 
signed  by  both  the  Assistant  Secretary  of  Defense  (l&L)  and'  the  Director  of  Defense  Re- 
search and  Engineering,  guidelines  for  a test  program  have  been  provided  tor  trial-basis 
use.  These  guidelines  are  similar  to  those  published  by  the  Air  force  in  July  |d"4  in 
Interim  U uid clincs  Reliability  Improve mcnt  Warranty  (RIW) . 

An  equipment  acquisition  contract  having  an  RIW  provision  specifies  equipment 
characteristics  for  form,  fit,  and  function  only,  and  allows  the  contractor  freedom  of  dv 
sign  as  well  as  freedom  to  change  design  without  intervention  of  all  the  usual  government 
configuration-management  constraints,  but  within  government  requirements  for  preserving 
configuration  control  the  RIW  provision  stipulates  that  the  contractor  will,  lor  a cvi 
tain  length  of  time  or  number  of  operating  hours,  repair  and  or  replace  the  units  m se  \ 
ice  upon  occasion  of  certain  defined  failures. 


A firm  fixed-price  contract  is  used,  one  price  including  both  acquisition  and  fol- 
low-on servicing.  Once  a fixed  price  is  established  for  the  contract,  the  actual  profit 
realized  by  the  contractor  depends  on  the  equipment’s  reliability  and  maintainability 
in  service  use,  plus  any  improvements  that  he  can  make  in  R&M  during  the  warranty 
period  to  keep  the  number  and  cost  of  repairs  as  low  as  possible.  In  this  connection. 
the  terms  and  commitments  required  of  the  contractor  by  the  RIW  provision  must  re- 
sult in  a reasonable  balance  between  his  risks  and  the  degree  of  incentive  (profits) 
needed  to  achieve  the  primary  goal  of  system/equipment  availability.  Such  considera- 
tion must  include  the  uncertainties  of  future  support  costs  and  the  resultant  risks  to  both 
contractor  and  government. 

The  RIW  is  not  the  same  as  a maintenance  contract  and  does  not  require  the  con- 
tractor to  provide  routine  periodic  upkeep,  regulation,  adjustment,  cleaning,  or  other  nor- 
mal upkeep.  Government  personnel  accomplish  these  jobs.  The  RIW  also  does  not  cover 
components  of  the  warranted  item  which  usually  need  replacement  under  normal  use;  such 
items  may  be  covered  by  separate  provisions  in  the  contract,  but  cannot  be  included  in  the 
RIW  provision. 

ADVANTAGES 

When  a new-procurement  item  meets  criteria  for  RIW  application  (to  be  discussed), 
the  use  of  the  RIW  offers  the  following  advantages  to  the  government: 

1.  Contractor  has  responsibility  and  incentive  for  improving  field  reliability. 

2.  Financial  commitment  for  logistic  support  is  known  and  constant  during 
a 3-5  year  period  of  possible  economic  inflation. 

3.  Life-cycle  cost  approach  can  receive  greater  emphasis. 

4.  Contractor  has  responsibility  for  configuration  management  of  field  units 
and  for  keeping  all  units  to  same  configuration. 

5.  Contractor  has  an  incentive  to  introduce  design  and  production  changes  that 
will  increase  MTBF  and  result  in  reliability  growth. 

6.  Contractor  has  an  incentive  to  introduce  design  and  production  changes  that 
will  reduce  labor  hours  and  materials  needed  for  field  repairs. 

7.  Minimal  initial  support  investment  is  required. 

8.  Requirements  for  skilled  military  maintenance  and  support  manpower  are  re- 
duced. 

Use  of  the  RIW  offers  the  following  advantages  to  the  contractor: 

1.  Increased  profit  potential  when  field  MTBF  is  increased  above  contractual 
MTBF. 

2.  Guaranteed  multiyear  business. 

3.  More  familiarity  with  operational  reliability  and  maintainability  characteris- 
tics. which  is  advantageous  in  obtaining  additional  government  contracts. 

Benefits  obtained  from  the  RIW  concept  can  be  maximized  only  if  prospective 
contractors  are  notified  early  in  the  development  period  that  the  government  intends  to 
consider  including  such  warranty  provision  in  the  production  contract.  Otherwise,  the 
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contractor  will  not  have  the  incentive  intended  by  the  RIW  to  give  necessary  attention  to 
R&M  at  the  time  of  initial  design,  the  outcome  of  which  has  significant  effect  on  eventual 
MTBF  and  repair  costs. 

USING  THE:  RIW 

The  use  of  RlWs  is  limited  to  certain  acquisitions  of  equipment  that  satisfy  ( I ) sti- 
pulations by  ASPR  for  warranty  use  in  general,  (2)  specific  criteria  for  RIW  use.  ( 3 > cer- 
tain funding  requirements,  and  (4)  conditions  relating  to  elements  that  must  be  contained 
in  the  RIW  clause  of  a contract.  These  sets  of  conditions  are  presented  in  the  following 
paragraphs. 

ASPR  REQUIREMENTS  FOR  GENERAL  WARRANTY  USF. 

ASPR  1-3 24.3b  lists  the  following  factors  that  must  be  considered  in  deciding 
whether  a warranty  clause  of  any  kind  is  to  be  used  in  a contract. 

1.  Nature  of  the  item  and  its  end  use 

2.  Cost  of  the  warranty  and  degree  of  price  competition  as  it  may  affect  this 
cost 

3.  Criticality  of  meeting  specifications 

4.  Damages  to  the  government  that  might  be  expected  to  arise  in  the  event  of 
defective  performance 

5.  Cost  of  correction  or  replacement,  either  by  the  contractor  or  another  source, 
in  the  absence  of  a warranty 

(->.  Administration  cost  and  difficulty  of  enforcing  the  war.anty 

7.  Ability  to  take  advantage  of  the  warranty,  as  conditioned  by  storage,  time, 
distance  of  the  using  agency  from  the  source,  or  other  factors 

S.  Operation  of  the  warranty  as  a deterrent  against  deficiencies 

4.  The  extent  to  which  government  acceptance  is  to  be  based  upon  contractor 
inspection  or  quality  control 

10.  Whether  because  of  the  nature  of  the  item  the  government  inspection  sys- 
tem would  not  be  likely  to  provide  adequate  protection  without  a warranty 

1 1 . Whether  the  contractor’s  present  quality  program  is  reliable  enough  to  pro- 
vide adequate  protection  without  a warranty 

12.  Reliance  on  "brand  name"  integrity 

13.  Whether  a warranty  is  regularly  given  for  a commercial  component  of  a more 
complex  end  item 

14.  Criticality  of  item  for  protection  of  personnel  or  property;  eg.  for  safety  in 
flight 

15.  The  stage  of  development  of  the  item  and  the  state  of  the  art 

lb  Customary  trade  practices 
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SPECIFIC  (RITFRIA  FOR  RIW  USI 


Currently  there  are  three  sources  for  criteria  which  establish  the  practicability  or 
advisability  of  using  the  R1W.  They  are  the  joint  memorandum  of  ASDtl&l  l/DDR&l  . 
the  Air  Force  interim  guidelines  document  for  RIWs:  and  a tabulation  of  criteria  devel- 
oped by  ARINC  Research  Corporation  for  the  Rome  Air  Development  Center  All  three 
cover  essentially  the  same  points.  The  ARINC  tabulation,  however,  is  more  precise  and 
definitive  and  is  presented  here  as  table  XV-I. 

Three  broad  areas  of  consideration  are  to  be  found  in  the  tabulated  criteria 
procurement  factors,  equipment  characteristics,  and  application  factors.  I ach  is  equally 
important  in  accepting  or  rejecting  RIW  use.  The  individual  factors  within  each  area, 
however,  do  vary  in  importance,  and  have  been  ranked  as  follows: 

"1”  factors  are  of  major  importance.  Failure  to  meet  the  stated  criterion  could 
be  grounds  for  not  using  the  RIW. 

“2"  factors  are  of  secondary  importance.  Failure  to  meet  the  criterion  id  one 
of  these  factors  will  generally  not  be  sufficient  in  itsell  for  rejecting  the 
RIW.  but  a combination  of  "2”  factors  could  be. 

"3"  factors  are  of  minor  importance.  Failure  to  meet  these  criteria  is  generally 
not  considered  serious,  but  may  require  special  consideration  in  structuring 
the  warranty  or  the  administrative  procedures. 

It  should  be  emphasized  that  the  criteria  in  table  XV-I  are  to  be  used  qualitatively 
with  an  awareness  of  the  extra  effort  and  cost  required  of  both  the  government  in  request- 
ing and  the  contractor  in  proposing  the  RIW  provision.  That  is.  there  should  be.  before 
requesting  contractors'  proposals,  a reasonable  certainty,  based  on  cost  analysis  as  well 
as  the  criteria  factors,  that  the  RIW  will  be  employed,  even  though  a thorough  economic 
analysis  of  the  use  cannot  be  made  until  RIW  price  and  implementation  proposals  are  re- 
ceived from  the  bidding  contractors.  To  decide  on  the  basis  of  cost  whether  or  not  the 
RIW'  may  be  a good  course  of  action,  there  must  be  a MTBF  figure  on  which  to  base  a 
probable  price  for  contractor  development/production  and  support  (acquisition  with  RIW  I. 
and  then  this  price  must  be  compared  with  the  price  of  a development /production-only 
contract  plus  the  probable  cost  of  support  by  the  government  without  the  RIW  fins  may 
be  undertaken,  as  indicated  in  figure  XV-I.  by  plotting  costs  vs  a range  ol  MI’BF  levels 
for  development/production  (curve  A)  and  operation/maintenance  (curve  ID.  which  curves 
when  added  provide  rough  figures  for  a life-cycle  cost  (curve  O.  The  vertical  width  of 
the  curves  in  figure  XV-I  represents  the  uncertainty  of  the  costs  associated  with  achieving 
and  supporting  the  various  levels  of  reliability.  The  use  of  life-cycle  cost  (ICO  models 
can  assist  in  this  evaluation  process  when  sufficient  data  are  available 

When  judgment  has  been  made  for  the  use  of  the  RIW  provision  m the  RI  P con 
tract,  the  RI  P is  worded  to  indicate  that  the  RIW  provision  may  or  may  not  be  included 
m the  contract,  depending  on  responding  proposals  and  what  they  have  to  say  about  pro- 
posed M I BI  and  cost. 

Upon  receipt  of  bidders'  proposals,  further  cost  analysis  is  made  to  determine 
whether  the  use  of  the  RIW  would  be  cost-effective.  The  life-cycle  cost  curve  developed 
prior  to  RI  P release  such  as  the  one  shown  in  figure  XV-1C  can  be  compared  with 
the  total  fixed-price  bids  in  the  proposals  and  their  respective  proposed  M I HI  s It  a 
cost  proposal  is  below  the  lower  hound  of  the  cost  curve,  the  price  is  right  and  use  ol 
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Procurement  Factors 

P ! The  procurement  is  to  be  on  a The  provisions  of  ASPR  provide  that  contractor 

fixed-price  basis.  warranty  expenses  are  admissible  under  a cost- 
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P.7  Analysis  of  warranty  price  versus  If  warranty  is  an  option,  the  final  decision 

organic  repair  costs  is  possible.  will  be  made  when  contractor  warranty  price 

bids  are  received.  Evaluation  of  such  bids  is 
best  made  when  compared  to  equivalent  costs 
under  organic  maintenance. 


E.3  - Unit  is  field  testable.  Since  compliance  with  most  warranties  will  re- 

quire the  failed  unit  to  be  returned  to  the  vendor, 
it  is  important  that  the  unit  be  field  testable  to 


1 = Major;  2 = Secondary;  3 = Minor 


h.8  - Unit  should  have  level  of  Delicate  units  highly  subject  to  failure  from 

ruggedization.  handling  and  shipping  may  lead  to  unacceptably 

high  rate  of  occurrence  of  warranty  exclusions 
for  mistreatment  or  abuse.  In  such  cases,  special 
shipping  containers  may  be  required. 
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I = Major;  2 = Secondary;  3 = Minor 


the  RIW  is  definitely  indicated,  if  the  proposal  is  above  the  upper  bound,  the  RIW  should 
not  be  considered.  In  between,  further  evaluation  must  be  made.  The  lair-price  question 
here  is  helped  considerably,  as  is  achievable  reliability,  by  the  fact  of  competition  among 
bidders  for  a fixed-price  contract.  Overestimation  of  expected  reliability  by  a bidder  will 
tend  to  increase  his  total  price  and  reduce  his  chance  of  being  accepted,  whereas  under- 
estimation for  the  purpose  of  improving  profit  may  lose  him  the  contract  as  well. 

A bidder  may  be  inclined  to  lump  the  costs  associated  with  the  RIW  into  his  unit 
price.  But.  in  order  to  make  an  accept/reject  decision  on  the  use  of  the  RIW.  the  actual 
price  proposed  for  the  RIW  must  be  known.  Bidders  therefore  must  be  required  to  sep- 
arately price  the  RIW  provision  so  that  a clear  comparison  may  be  made  with  the  govern- 
ment's cost  estimate.  The  accept/reject  decision  is  based  in  no  small  part  on  the  differ- 
ence between  the  RIW  support  costs  and  the  support  costs  the  government  would  incur 
if  the  equipment  were  purchased  without  an  RIW.  The  warranty  should  normally  cost 
4-6%  of  the  operational  unit  acquisition  cost  per  year. 

FUNDING  R EQ U 1 R E M E NTS 

To  clarify  the  types  of  funds  to  be  used  for  procurements  having  the  RIW  pro- 
vision. the  following  funding  policy  guidelines  have  been  authorized  for  use  by  OASD(C) 
and  OAGC(FM): 

1.  RlWs  shall  be  funded  from  the  same  appropriation  as  the  acquisition.  That 
is,  the  RIW  shall  be  paid  from  the  procurement  or  RDT&E  appropriation 
of  the  service  or  agency  concerned,  depending  on  from  which  of  the  appro- 
priations the  acquisition  is  funded. 

2.  The  RIW  price  shall  be  part  of  the  fixed-price  contract,  and  payment  to  the 
warrantor  for  RIW  portion  shall  not  be  made  differently  from  payment 
under  the  remaining  portion  of  the  contract  except  that  payment  for  the 
RIW  may  be  delayed  until  delivery  or  relinquishment  of  production  control 
of  the  item  by  the  warrantor. 

In  addition,  to  maintain  the  distinction  between  an  RIW  and  a service  contract 
covering  normal,  periodic  maintenance,  the  following  requirements  must  be  satisfied: 

3.  The  warranty  period  on  the  item(s)  shall  begin  after  manufacture  and  upon 
delivery  or  relinquishment  of  production  control  by  the  warrantor. 

4.  The  RIW  shall  require  the  warrantor  to  repair  or  replace  the  warranted  item 
upon  failure. 

5.  The  RIW  shall  not  include  requirements  for  the  warrantor  to  provide  nor- 
mal upkeep,  cleaning,  adjusting,  regulating,  or  other  periodic  maintenance 
accomplished  whether  or  not  failure  occurs. 

6.  The  RIW  shall  exclude  components  of  the  warranted  item(s)  which  normally 
require  replacement  during  the  period  of  the  warranty  (such  as  filters  and 
light  bulbs).  These  components  may  be  accounted  for  by  separate  provi- 
sions in  the  contract  consistent  with  current  laws  and  regulations. 
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KSSENTIAL  ELEMENTS  IN  THE  RIW  CLAUSE 


Because  RIW  provisions  must  be  tailored  to  the  particular  item  being  warranted,  a 
standard  RIW  clause  is  not  contained  in  the  ASPR.  However,  certain  elements  should  be 
considered  for  inclusion  in  an  RIW  clause.  These  elements,  which  are  discussed  below, 
concern  (1)  the  statement  of  contractor  warranty.  (2)  contractor  obligations.  (3)  govern- 
ment obligations.  (4)  miscellaneous  requirements,  and  (5)  data  requirements. 


STATEMENT  OF  CONTRACTOR  WARRANTY 

1.  Term.  State  the  length  of  time  the  warranty  will  be  in  effect.  Usually  this 
should  cover  usage  (operating  hours)  and  calendar  time  (generally  3 to  5 years), 
whichever  occurs  first. 

2.  Objective/scope.  State  the  primary  objective  of  the  warranty;  that  is,  to 
motivate  the  contractor  to  design  and  produce  equipment  that  is  more  relia- 
ble and  less  costly  to  repair.  If  there  is  a specified  reliability  requirement. 

it  should  be  clearly  set  forth. 

3.  Failure.  Define  the  failure(s)  which  will  require  the  cr  .tractor  to  repair  or 
replace  a failed  item  at  no  change  in  the  contract  price,  when  the  item  is 
returned  to  him. 

4.  Exclusions.  State  the  conditions  and  actions  associated  with  repair/replace- 
ment which  are  specifically  excluded  under  the  warranty  such  as  items  lost 
or  damaged  due  to  fire,  explosion,  packing,  and  shipping. 

5.  Shipping  costs.  State  who  pays  the  costs  of  shipping  the  failed  units  to  the 
contractor,  the  government  or  the  contractor. 

6.  Price.  State  that  a price  breakout  for  the  warranty  should  be  included  (along 
with  the  total  price)  to  allow  the  government  to  determine  the  cost  of  the 
RIW. 

\ 

CONTRACTOR  OBLIGATIONS 

1.  Warranty  markings.  State  that  the  contractor  shall  be  required  to  prominently 
display  the  following  information  on  the  face  of  the  unit;  that  the  item  is 
warranted;  the  warranty  period;  and  actions  to  take  if  the  unit  fails  during 
the  warranty  period. 

2.  Turnaround  time.  State  the  turnaround  time,  and  the  appropriate  adjustments 
or  other  considerations  to  be  exacted  if  the  contractor  exceeds  the  number 

of  days  so  specified.  A contract  turnaround  time  should  be  defined  as  the 
period  between  the  date  the  unit  is  received  by  the  contractor  for  repair/ 
replacement  and  the  date  when  the  repaired/replacing  unit  is  shipped  by  the 
contractor  to  the  government. 

3.  Records.  State  that  the  contractor  shall  maintain  records,  by  serial  number, 
for  each  unit  under  warranty,  and  shall  make  such  records  available  to  the 
government  upon  request. 


XV  14 


GOVERNMENT  OBLIGATIONS 


1.  Containers.  State  whether  or  not  the  government  will  supply  special  containers 
for  transporting  units  to  and  from  their  destinations  for  the  life  of  the  warranty. 

2.  No-cost  modifications.  State  the  procedures  for  submittal  of  contractor- 
initiated  no-cost  Engineering  Change  Proposals  (designed  to  improve  the 
item’s  reliability /maintainability ).  The  contractor  should  be  advised  that  such 
ECPs  will  be  subject  to  government  approval. 


MISCELLANEOUS 

1.  Inspection.  State  the  extent  of  both  government  and  contractor  inspection 
to  be  required. 

2.  Disposition.  State  that  each  unit  returned  to  the  contractor  which  is  con- 
sidered to  be  nonrepairable  shall  be  disposed  of  by  the  contractor  as  directed 
by  the  government.  Also,  state  the  manner  of  disposition  of  the  unused 
portion  of  the  warranty  for  such  nonrepairable  unit  as  well  as  for  other 
similar  instances  such  as  when  the  government  has  certified  that  a unit  has 
been  lost. 

3.  Notification.  State  the  requirements,  including  time  limits,  for  both  the 
government  and  contractor  to  notify  each  other  of  deficiencies  in  a unit. 

4.  Unverified  failures.  State  whether  or  not  the  contractor  will  be  compensated 
for  the  cost  of  testing  units  returned  to  him  under  the  warranty  when  the 
tests  reveal  no  discrepancies. 

5.  Adjustments.  State  the  circumstances,  if  any,  under  which  the  government 
is  authorized  to  make  adjustments  to  units  under  the  warranty. 


DATA  REQUIREMENTS 

1.  Contractor  warranty  data.  State  that  the  contractor  shall  establish  and  main- 
tain a data  system  that  will  provide  information  on  the  repair  record  of  each 
unit,  analysis  of  unit  failure,  number  of  units  returned,  turnaround  and  pipe- 
line times  of  units  returned,  remaining  warranty  coverage,  etc. 

2.  Government-developed  data.  State  that  the  government  will  be  required  to 
provide,  in  a timely  manner,  available  government-generated  operation  and 
maintenance  data  pertaining  to  the  warranted  units. 

SUMMARY  OF  RIW  USE 

Even  to  consider  the  use  of  the  RIW  in  a procurement  contract  depends  on  whether 
the  item  is  expected  to  have  moderate  to  high  support  costs.  If  so.  then  the  item  must 
be  amenable  to  being  specified  only  in  terms  of  form,  fit,  and  function,  without  the  need 
for  detail  design  constraints.  When  these  conditions  prevail,  then  the  four  measures  of 
suitability  discussed  above  can  be  applied.  And  if  the  item  measures  up,  the  project  mana- 
ger has  the  green  light  to  go  ahead  with  the  RIW. 


' 
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OTHER  R&M  COST  REDUCTION  WARRANTIES 


Three  other  types  of  R&M  cost-reduction  warranties  are  presented.  They  are  re- 
ferred to  as  Mean  Time  Between  Failure.  Equipment  Turnaround  Time,  and  Maintenance 
Cost  names  that  indicate  the  nature  of  the  guarantee  being  purchased.  They  are  warranties 
with  which  the  airline  industry  has  had  some  success,  but  with  which  the  government  has  had 
littk  experience.  Consequently,  their  use  as  allowed  by  ASl’R  must  be  carefully  considered, 
just  as  the  use  of  the  Reliability  Improvement  Warranty  must  be  caretully  considered  and 
their  clauses  must  be  tailored  to  the  particular  procurement  requirements.  Clauses  that  have 
been  used  by  the  airline  industry  are  presented  for  each  ot  these  warranties  as  examples  only 
of  the  elements  to  be  considered  for  inclusion  in  government  clauses. 


MEAN  TIME  BETWEEN  FAILURES  (MTBF)  WARRANTIES 

Under  this  type  of  warranty  the  contractor  guarantees  that  the  equipment  will  a- 
chieve  a stated  MTBF.  If  the  equipment  fails  to  do  so,  the  contractor  must  provide  labor 
and  materials  to  modify  the  equipment  to  meet  the  MTBF  requirements.  In  essence,  this  is 
another  way  to  obtain  a required  level  of  reliability. 

An  MTBF  warranty  can  reduce  infant  mortality  because  it  gives  the  contractor  an 
incentive  to  test  the  equipment  before  delivering  it  to  the  government.  On  the  other  hand, 
historical  information  on  the  equipment  must  be  made  available  to  determine  an  equitable 
and  realistic  price.  The  contract  also  must  contain  definitions  for  various  classes  of  failures 
which  must  be  mutually  satisfactory  to  both  parties.  Also,  conditional  MTBF  warranties 
can  lead  to  disputes;  an  unconditional  warranty  under  which  all  agreed-upon  failure  condi- 
tions are  rectified  is  more  desirable,  but  also  more  expensive. 


EXAMPLES  OF  CLAUSES 

1 . Each  manufacturer  guarantees  that  the  Equipment  covered  under  this  agree- 
ment shall  achieve  the  Mean  Time  Between  Failure  (MTBF)  guarantees  as  set 
forth  in  Attachment  XXX.  a copy  of  which  is  attached  hereto  and  made  a 
part  hereof. 

2.  Buyer  shall  provide  Equipment  failure  data  from  the  date  the  Equipment  enters 
the  Buyer's  service.  The  data  shall  be  sufficient  to  determine  MTBF  and  any 
additional  equipment  required. 

3.  Equipment  provisioning  shall  be  determined  by  Buyer  and  shall  be  based  upon 
the  MTBF  guarantee  set  forth  in  Attachment  XXX  as  modified  by  other  pro- 
gram factors  determined  by  Buyer.  MTBF  guarantees,  set  forth  in  Attachment 
XXX.  for  the  purpose  of  this  program,  shall  be  divided  into  three  periods; 

(a)  Initial  first  24  months;  (b)  Interim  25th  to  42nd  months;  and  <c)  Fi- 
nal 43rd  and  subsequent  months. 

4.  Support  is  to  be  based  upon  the  data  provided  by  Buyer  in  accordance  with 
paragraph  2 above,  and  such  data  shall  be  used  to  compute  any  additional  e- 
quipment  required.  Such  equipment  shall  be  made  available  on  a no-charge  con- 
signment basis. 
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MTBF  measurements  shall  be  based  upon  a monthly  measurement  correspond- 
ing to  a 3-month  moving  average.  The  Manufacturer's  obligation  under  the 
MTBF  Guarantee  Program  shall  commence  on  the  date  the  Fquipment  enters 
Buyer’s  service  and  shall  terminate  when  the  respective  MTBF’  guarantees  set 
forth  in  Attachment  XXX  are  achieved  over  IS  consecutive  monthly  measure- 
ments commencing  no  earlier  than  the  43rd  month  after  introduction  of  the 
Equipment  into  Buyer’s  service. 

The  specific  provision  for  measuring  the  MTBF  is  as  hereinafter  set  forth: 

a.  Calculation  of  MTBF:  The  MTBF  shall  be  calculated  by  the  application 
of  the  following  formula: 


Cumulative  Number  The  Number  of 

of  Hours  of  Equip-  X Equipments 
Ment  Operation 

Total  Cumulative  Number  of  Chargeable  Failures 


b.  Definition  of  Failure:  The  following  failure  definitions  and  conditions 
shall  apply: 

( 1 I Confirmed  Failure:  Any  Equipment  removed  from  a platform  for 
suspected  failure  shall  be  deemed  a confirmed  failure  when,  upon 
being  subjected  to  test  in  the  condition  removed,  it  is  unable  to 
pass  the  test  I >r  the  Equipment  specified  by  Manufacturer's  Over- 
haul Manual  supplied  to  Buyer  or  other  mutually  agreeable  test  pro- 
cedure. The  specified  test  must  be  comparable  in  scope  to  Manu- 
facturer's acceptance  test  for  production  equipment.  Tests  may  be 
performed  in  Buy  er's  facilities  or  those  of  its  approved  designee. 

(2)  Irrelevant  failures  Irrelevant  failures  shall  not  be  counted  in  the 
M I BE  determination.  Irrelevant  failures  are  defined  as  follows: 


(a)  \ failure  caused  by  a condition  external  to  the  Equipment, 

such  as  improperly  supplied  power,  improper  interconnecting 
wiring,  or  improper  operation  of  the  Fquipment. 

ibi  I lie  l.iilurc  is  a dependent  (secondary)  failure  resulting  from 
an  independent  (primary  ) failure  within  the  same  Equipment, 
provided  that  the  independent  (primary)  failure  is  specified. 

\ dependent  failure  occurring  in  a separate  piece  of  equipment 
from  the  Equipment  in  which  the  primary  confirmed  failure 
occurred  shall  be  considered  as  a confirmed  failure 


(3)  Additional  Requirements  At  all  times  while  in  Buyer's  possession, 
the  Equipment  shall  be  subjected  to  an  em  •ronment  within  Specifi- 
cation requirements  f ailures  which  occur  as  a result  of  an  exposure 
to  an  environment  in  excess  of  that  specified  shall  not  count  in  the 
\l  I HI  determination.  Failures  resulting  from  accident  or  improper 
maintenance  shall  not  count  in  the  M I BE  determination.  Operation 
and  maintenance  procedures  shall  be  in  accordance  with  the  appli- 
cable operating  and  maintenance  manuals 
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7.  a.  In  the  event  the  MTBF  calculated  for  any  Hquipment  in  operation  in  a 
calculation  period  is  less  than  the  guaranteed  MTBF.  Manufacturer  shall 
re-engineer  and  modify  such  Hquipment  as  required  to  meet  the  guaranteed 
MTBF.  at  no  charge  to  Buyer,  and  consign  additional  Hquipment  at  no 
charge  based  on  the  following  formula: 


n 


= KN 


( M-tii  l 
Min 


where 

n = Maximum  number  of  additional  Hquipment  to  be  consigned  to  Buyer 
under  MTBF  Guarantee  Program.  This  number  shall  be  rounded  to 
the  nearest  whole  number,  but  not  less  than  1 and  shall  not  exceed 
100'  of  Hquipment  installed  in  Buyer's  platlorms  as  ol  the  date  ot 
MTBF  calculation. 

K = 5000 

N = Total  number  of  Hquipment  installed  in  Buyer's  platlorms  as  ol  the 
date  of  MTBF  calculation. 

M = Guaranteed  MTBF  for  the  Hquipment. 

m = Calculated  average  MTBF  for  the  Hquipment. 


h.  Failure  classification  shall  be  mutually  agreeable  to  Manufacturer  and 
Buyer.  If  no  agreement  can  be  reached,  the  failed  Hquipment  shall  be 
subject  to  failure  analysis  prior  to  classification. 

e.  If  additional  consignment  Hquipment  is  required  to  be  furnished  by  Manu- 
facturer to  Buyer  hereunder.  Manufacturer  shall  ship  such  Hquipment  to 
Buyer  not  later  than  I week  after  completion  of  the  MTBF  calculation  by 
Buyer.  Buyer  shall  notify  Manufacturer  if  the  indicated  number  of  con- 
signment Hquipment  exceeds  Buyer’s  requirements,  in  which  case.  Manu- 
facturer shall  be  obligated  to  supply  only  that  quantity  required  by  Buyer. 
Manufacturer  agrees  to  furnish  such  additional  Hquipment  notwithstanding 
the  possible  existence  of  any  disagreement  as  to  failure  classification. 

8.  Return  of  Consigned  Equipment:  Any  Equipment  consigned  under  the  provi- 
sions of  paragraph  7 above  shall  be  returned  to  Manutacturer  by  Buyer  not 
later  than  lK)  days  after  an  MTBF  calculation.  Buyer  shall  have  the  fiexibilitv 
of  replacing  any  consigned  Equipment  with  similar  Hquipment. 


EQUIPMENT  TURNAROUND  TIME  WARRANTIES 

Under  the  terms  of  this  warranty  the  manufacturer  guarantees  to  repair  Ins  delivered 
equipment  within  a stipulated  average  turnaround  time.  Normally,  it  is  assumed  that  the 
same  unit  will  be  returned  to  the  government  after  repair,  but  allowances  can  be  made  to  re- 
place the  unit  with  another  like  unit  to  expedite  turnaround  time. 

This  type  of  warranty  is  especially  suited  to  depot-level  or  manufacturer  repair  sit- 
uations. Only  a limited  area  of  concern  is  covered,  however.  Other  guarantees  would  have 
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to  bo  used  to  cover  reliability  as  well  as  the  costs  of  maintenance  and  repair.  This  warranty 
woidd  have  to  be  tailored  it  it  was  to  be  used  for  shipboard  equipment,  which  should  not 
be  too  difficult. 


1 


1 XAMI'Ll  SOI  CLAIJS1  S 

I.  Manufacturer  guarantees  the  average  repair  turnaround  time  as  listed  in  Attach- 
ment XXX,  a copy  of  which  is  attached  hereto  and  made  a part  hereof,  for  the 
hquipment  covered  under  this  Agreement  requiring  repair  or  correction.  These 
turnaround  times  include  round-trip  shipping  time  between  Buyer’s  maintenance 
facilities  and  Manufacturer’s  repair  facility.  The  average  turnaround  time  shall 
be  calculated  on  a 3-month  moving  average.  In  the  event  Manufacturer  tails  to 
meet  the  average  turnaround  times  as  listed  in  Attachment  XXX.  Manufacturer 
shall  consign  additional  Equipment,  at  no  charge,  in  accordance  with  the  follow- 
ing formula: 

N = H (t  -T) 


where 

N = Number  of  additional  Equipment  to  be  consigned  to  Turnaround 

Time  Guarantee.  This  number  shall  be  rounded  to  the  nearest  whole 
number,  but  shall  not  be  less  than  1. 

R - Quantity  of  equipment  returned  to  Manufacturer  for  repair,  in 
units  per  week,  calculated  on  a 3-month  moving  average. 

T = Guaranteed  repair  turnaround  time  in  weeks. 

t = Calculated  average  repair  turnaround  time  in  weeks. 

2.  The  above  assumes  that  the  same  serial  numbered  equipment  shall  be  returned 
to  Buyer  as  received.  However.  Manufacturer  shall  have  flexibility  to  replace 
the  failed  equipment  with  other  similar  equipment,  if  this  will  expedite  repair, 
provided  that  the  number  of  hours  on  such  other  equipment  shall  not  exceed 
the  number  on  such  failed  equipment 


MA1NTCN  AN(T  COST  VVARR  \NTICS 

Under  this  warranty  the  contr.  ctor  guarantees  that  the  labor  and  material  costs  for 
the  government  to  maintain  the  equipment  will  not  exceed  a specified  amount.  If  this  a- 
mount  is  exceeded,  the  contractor  will  reimburse  the  government  for  the  excess. 

Use  of  this  type  of  warranty  allows  the  government  to  use  its  own  maintenance  per- 
sonnel. The  warranty  could  be  practicable  lor  use  with  shipboard  equipment.  However,  a 
maintenance  history  of  the  equipment  is  required  to  determine  an  equitable  and  realistic 
contract  price  and.  during  application,  thorough  maintenance-cost  records  must  be  kept. 
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1 XAMPl  I S Of  CLAUSES 


1.  Manufacturer  guarantees  that  the  maintenance  labor  and  material  cost  for 
Buyer  to  maintain  the  Equipment  covered  under  this  Agreement  shall  not 
exceed  the  respective  guaranteed  maintenance  labor  man-hour  costs  and 
maintenance  material  costs  set  forth  in  Attachment  XXX.  a copy  of  which 
is  attached  hereto  and  made  a part  hereof.  The  Maintenance  Cost  Guaran- 
tees  set  forth  in  Attachment  XXX  shall  commence  with  delivery  of  the 
first  unit  to  Buyer  and  shall  terminate  8 years  thereafter. 

2.  At  periodic  intervals,  but  not  less  than  once  each  year,  after  delivery  of 
the  first  unit  to  Buyer.  Buyer  shall  report  its  recorded  maintenance  labor 
man-hour  costs  and  maintenance  materials  costs  for  Manufacturer's  Equip- 
ment.  In  the  event  such  periodic  maintenance  labor  man-hour  costs  or 
maintenance  materials  costs  are  in  excess  of  those  set  forth  in  Attachment 
XXX  hereto.  Manufacturer  agrees  that  it  shall  provide  at  no  additional 
cost  to  Buyer  and  at  the  request  of  Buyer,  the  following: 

(a)  Technical  service  support  by  qualified  personnel 
( b > Corrective  engineering  design  changes 

(c>  Modification  of  all  applicable  I quipment  covered  by  this  Agreement 

( d ) Reimbursement  to  Buyer  for  all  costs  incurred,  above  those  set  forth 
herein,  on  an  annual  basis 

Such  service  as  outlined  above  shall  be  performed  by  Manufacturer  at  Buyer's 
request  in  order  to  bring  the  direct  maintenance  labor  man-hours  and  main- 
tenance material  costs  within  the  Maintenance  Cost  Guarantees  set  torth  in 
Attachment  XXX. 

3.  This  program  pertains  only  to  maintenance  labor  and  material  costs  for  Man- 
ufacturer's Tquipment.  All  labor  and  material  costs  shall  exclude  acts  of 
omission  of  Buyer,  acts  of  God  or  a third  party.  Buyer  modifications  un- 
related to  improvements  in  operating  cost  of  Manufacturer's  equipment,  con- 
sumable fluids,  or  outside  maintenance  burden  and  profit  charges. 


GGVeRNMENT  STUDY  RECOMMENDATIONS  PERTAINING  TO  WARRANTY  US1 

The  following  recommendations  are  excerpted  from  appendix  B.  and  serve  to  sum- 
mari/e this  section. 

1.  Use  long-term  contractor  maintenance  warranties  to  motivate  the  contractor 
to  design  for  minimum  life-cycle  cost 

2.  To  overcome  the  potential  problem  of  spare-parts  stocking  and  field  repair 
of  multiple  equipment  configurations,  make  use  of  depot  repair  or  supplier 
maintenance  under  warranty.  In  the  field,  replace  rather  than  repair  tailed 
replaceable  units.  Include  warranty  requirements  when  initiating  develop- 
ment. 


3.  Institute  a cost  accounting  system  that  will  afford  visibility  of  the  mainte- 
nance process  and  make  possible  realistic  cost  comparisons  between  unlit  ary 
and  industrial  maintenance. 

4.  Establish  alternative  sources  of  maintenance,  including  the  maximum  feasible 
amount  of  contractor  maintenance,  to  foster  competition  and  resultant  ef- 
ficiency in  the  maintenance  process  and  to  ensure  the  proper  utilization  ot 
scarce  military  personnel  in  the  present  zero-draft  environment. 

5.  1 xtend  the  application  of  long-term  contractor  maintenance  warranties  to 
military  electronic  procurements. 

6.  Make  known  the  intention  to  contract  for  maintenance  warranties  on  pro- 
duction equipment  at  the  time  development  is  initiated,  si'  that  the  con- 
tractor will  design  to  minimize  total  costs  of  production  and  warranty  main- 
tenance. 

7.  1 stablish  a warranty  review  group  within  OSI)  to  monitor  results  ot  trial 
applications,  to  determine  desirable  warrantv  contractual  formats,  and  to 
refine  the  categories  of  equipments  to  which  warranties  are  most  applicable 
and  for  which  warranties  are  most  effective. 

S.  Initially,  apply  long-term  contractor  maintenance  warranties  to  equipments 
whose  failed  units  can  be  replaced  in  the  lield  and  conveniently  returned 
to  the  contractor's  plant  or  base  for  repair,  or  to  which  the  contractor  can 
have  ready  access  for  field  repair,  such  as:  airborne  communications,  navi- 
gation. and  identification  equipment;  modular  radars;  vehicular  communica- 
tion sets;  complex  manpack  equipment  such  as  I ORAN  ( l>;  lorward-look- 
ing  infrared  (l'l  I K > systems;  and  domestic  communication,  data  processing, 
and  radar  installations. 

0.  The  government  should  defer  invocation  of  the  final  product  baseline,  as 

applicable  to  electronic  equipment,  until  field  reliability  obteclives  have  been 
achieved,  or.  in  the  case  of  equipment  under  contract  maintenance  warranty, 
until  the  warrantv  period  is  about  to  end  and  the  government  is  about  to 
take  over  maintenance  from  the  warrantor 

10.  Use  contractor  warranties  and  maintenance  to  reduce  the  need  lor  technical 
and  maintenance  manuals  and  provisioning  data 

TIN  \L  CONSIDERATIONS 

The  use  of  long-term  warranties  is  subject  to  the  same  tradeoff  considerations  that 
apply  to  other  forms  of  nonorganic  maintenance  Only  m rare  circumstances  can  organic 
maintenance  be  performed  without  compromising  the  warranty  I urthermore.  the  use  ot 
the  equipment  must  lend  itself  to  the  collection  ot  valid  lield  failure  data  and  to  a rea- 
sonable description  of  the  mission  and  use  environments.  In  short,  the  mission  must  be 
of  a character  that  allows  nonorganic  maintenance  with  reasonable  turnaround  time  It 
should  be  noted  that  nonorganic  maintenance  will  generally  require  higher  procurement 
quantities  to  till  supply  pipelines  and  equipment  pools,  for  mission-essential  equipments, 
these  pools  should  include  an  inventory  hedge  against  possible  disruptions  siuh  as  strikes. 
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TWO  PRINCIPAL  TYPES  OF  CONTRACTS 

Basically,  there  are  two  types  of  contracts  fixed  price  and  cost.  The  major  dis- 
tinction between  the  two  is  in  the  nature  of  the  seller’s  obligation.  Under  a fixed-price 
contract,  the  contractor  niust  produce  the  required  items  or  perform  the  services  for  the 
firm  fixed  price  or  within  the  ceiling  price  of  an  incentive  contract  or  he  is  subject  to  the 
penalties  provided  in  a Default  clause.  There  are  various  types  ol  fixed-price  contracts 
firm  fixed  price  (FFP),  fixed  price  with  escalation  (FPE),  and  fixed  price  incentive  (FPI). 

The  second  general  category  of  contracts  is  cost  reimbursement.  Under  a cost-type 
contract,  the  product  is  not  paid  for  on  the  basis  of  an  invoice  price;  rather  the  govern- 
ment pays  the  contractor’s  cost  of  material  and  labor  and  a portion  of  his  overhead  costs 
as  provided  in  Cost  Principles  cited  in  the  contract.  Cost-type  contracts  include  cost,  cost 
plus  fixed  fee,  cost  plus  incentive  fee,  and  eost  plus  award  fee. 

Under  a cost-type  contract,  the  contractor  agrees  to  use  his  best  efforts  to  com- 
plete the  contract  within  the  estimated  amount  provided  in  the  contract  but  has  no  obli- 
gation for  further  performance  when,  despite  his  best  efforts,  the  contract  is  not  fully 
performed  at  the  time  he  expends  the  funds  in  the  contract,  unless  the  ( ontracting  Officer 
increases  the  funds. 

A time  and  material  type  contract  is  a hybrid  form  under  which  the  contractor  is 
paid  a fixed  price  for  each  labor  hour,  which  includes  the  labor  costs,  indirect  expenses, 
and  profit,  plus  material  at  cost.  See  tabic  XVI-I  for  a summary  of  contract  types. 

Appendix  A of  this  chapter  is  a glossary  of  procurement  terms.  Appendix  B is  a NAV- 
MAT  instruction  on  filling  out  1)1)  Form  1423,  “Contract  Data  Requirements  List.  " 


BASIS  OF  SELECTION  OF  THE  TYPE  OF  CONTRACT 

The  major  consideration  in  the  selection  of  the  type  ot  contract  should  be  whether 
or  not  the  item  can  be  made  or  the  services  can  be  performed.  From  the  contractor’s 
viewpoint,  if  he  is  sure  that  he  can  make  the  item  and  can  secure  a price  which  will  ade- 
quately cover  contingencies  and  risks  in  the  contract,  then  a firm  fixed-price  contract  is 
best.  On  the  other  hand,  after  determining  that  the  contractor  can  reasonably  be  expected 
to  perform  the  contract,  the  Contracting  Officer  must  determine  whether  the  price  asked 
is  a reasonable  one  considering  the  risks  in  the  procurement.  II  the  Contracting  Officer 
finds  that  there  are  unusual  contingencies  present  which  the  contractor  has  taken  into  con- 
sideration in  establishing  his  price,  then  the  Contracting  Officer  must  negotiate  a price 
which  equitably  shares  the  risk  between  the  contractor  and  the  government,  or  he  should 
include  provisions  for  escalation,  with  or  without  incentive  provision,  which  will  allow 
him  to  recapture  the  contingencies  included  in  the  contractor  s price  il  they  do  not  occur. 

Where  the  contractor  is  not  certain  that  he  can  perform  the  contract  due  to  uncer- 
tainties in  the  contract  requirements  which  prevent  him  from  developing  a reasonable  esti- 
mate of  the  cost  of  performance,  or  where  the  contract  calls  for  advances  in  the  state  ot 
the  art  which  the  contractor  is  not  sure  he  can  achieve,  then  he  should  endeavor  to  secure 
a cost-type  contract.  On  the  other  hand,  under  a cost-type  contract,  the  C ontracting 
Officer  has  no  assurance  of  the  success  of  work  being  performed  and  no  idea  of  how  much 
it  will  eventually  cost.  This  type  of  contract  can  only  be  used  where  the  Contracting 
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TABU  XVI  I.  CONTRACT  TYPES. 
contract  types 


Officer  determines  that  it  is  likely  to  be  less  costly  than  other  methods,  or  that  it  is  imprac- 
ticable to  secure  supplies  or  services  of  the  kind  or  quality  required  without  the  use  of  such 
types  of  contracts. 

If  he  has  the  option,  the  contractor  should  base  selection  of  the  type  of  contract 
on  the  knowledge  acquired  during  the  preparation  of  the  cost  estimate  associated  with  the 
proposal.  The  government  makes  its  decision  on  the  basis  of  competitive  prices  or  the 
cost  or  price  analysis  of  the  contractor’s  proposal.  In  addition,  a major  factor  in  deter- 
mining the  type  of  contract  and  its  terms  is  the  relative  bargaining  strength  and  negotiating 
ability  of  both  sides.  With  reference  to  this,  ASPR  3-401.  Types  of  Contracts,  provides  as 
follows: 

a.  To  provide  the  flexibility  needed  in  the  purchase  of  the  large  variety  and  vol- 
ume of  military  supplies  and  services,  a wide  selection  of  types  of  contracts 
is  available  to  the  contracting  parties.  The  respective  contract  types  vary  as 
to  (i)  the  degree  and  timing  of  responsibility  assumed  by  the  contractor  for 
the  costs  of  performance,  and  (ii)  the  amount  and  type  of  profit  incentive 
offered  the  contractor  to  achieve  or  exceed  specified  standards  or  goals.  With 
regard  to  degree  of  cost  responsibility,  the  various  types  of  contracts  may  be 
arranged  in  order  of  decreasing  contractor  responsibility  for  the  costs  of 
performance.  At  one  end  is  the  firm  fixed-price  contract  under  which  the 
parties  agree  that  the  contractor  assumes  full  cost  responsibility.  At  the  other 
end  of  this  range  is  the  cost-plus-a-fixed-fee  contract  where  profit,  rather  than 
price,  is  fixed  and  the  contractor’s  cost  responsibility  is  therefore  minimal. 

In  between  are  the  various  incentive  contracts  which  provide  for  varying 
degrees  of  contractor  cost  responsibility,  depending  upon  the  degree  of  un- 
certainty involved  in  contract  performance. 

The  purpose  of  the  contractor’s  estimating  process  and  the  government’s  price  and 
cost  analysis  is  to  develop  some  idea  of  the  possible  range  of  costs  within  which  the  con- 
tract work  may  be  performed.  The  size  of  this  range  from  minimum  to  maximum  factored 
by  the  bargaining  position  will  determine  the  type  of  contract  used  and  therefore  the 
relative  cost  risk  of  the  contract  to  both  parties.  For  example: 


$ 1 00  000 
“Probable” 

Cost 

S95  000  3 1 05  000 


SO 5 000  SI  20  000 


$80  000  SI 40  000 


Firm  fixed  price 
Confidence  range  ±5'7 

Fixed  price  incentive 
Confidence  range  ±5'/  to  20'  < 

Cost  plus  incentive  fee  development 
Confidence  range  f20'  or  more 

Cost  plus  fixed  fee 
R & 1) 

Minimum  confidence  in  cost  and  term 


Establishing  the  range  of  possible  cost  is  not  easy.  Since  the  estimate  is  just  that, 
an  estimate,  conclusions  regarding  its  accuracy  can  be  no  more  than  similar  estimates. 
Consideration  of  the  following  factors  will  provide  guidelines  as  to  the  nature  ol  the  type 
of  contract. 
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1.  Nature  of  the  work.  In  general,  a high  ratio  of  development  to  fabrication 
will  tend  to  create  a high  degree  of  uncertainty. 

2.  Past  experience.  In  situations  where  both  the  contractor  and  the  government 
have  had  little  experience  in  estimating  costs  on  similar  work,  confidence  in 
the  estimate  will  be  low. 

3.  Time  available.  The  degree  of  confidence  in  the  estimate  increases  substan- 
tially if  adequate  time  is  available  for  careful  estimation. 

Usually,  the  request  for  proposal  will  specify  the  type  of  contract  which  the  cus- 
tomer wishes.  In  the  majority  of  cases,  however,  proposals  on  alternative  types  of  con- 
tracts will  be  considered.  Where  the  alternate  proposal  decreases  the  contractor's  share  of 
the  risk  for  example,  proposing  on  a cost-type  basis  when  the  customer  has  requested 
a fixed-price  proposal  it  may  seriously  allect  the  contractor's  chances  ol  winning  follow- 
ing is  a brief  description  of  types  of  contracts. 

FIRM  FIXED-PRICE  CONTRACT 

This  is  the  basic  type  of  contract.  In  formally  advertised  procurements,  the  firm 
fixed-price  contract,  with  or  without  provision  for  escalation,  is  the  only  type  that  may 
be  used.  In  negotiated  procurements.  ASPR  directs  that  it  be  used  unless,  under  the 
attendant  circumstances,  use  of  another  type  is  more  appropriate. 

lire  firm  fixed-price  contract,  as  the  name  implies,  is  an  agreement  by  the  contrac- 
tor to  furnish  designated  supplies  or  services  at  a specified  price  which  is  not  subject  to 
adjustment  in  the  light  of  performance  costs.  In  its  basic  form,  the  firm  fixed-price  con- 
tract carries  the  greatest  risk  and  offers  the  greatest  possibility  of  profit  or  loss  of  any 
type  of  contract.  The  contractor  cannot  collect  more  than  the  agreed  fixed  price  but  is 
entitled  to  receive  the  full  amount  of  the  fixed  price,  regardless  of  his  actual  performance 
costs.  This  type  of  contract  is  best  suited  for  procurements  where  reasonably  definite 
specifications  are  available,  price  competition  exists,  production  experience  is  present,  and 
costs  can  be  predicted  with  reasonable  certainty.  Examples  of  items  for  which  this  con- 
tract is  used  are  standard  commercial  items,  modified  commercial  items,  or  military  items 
for  which  adequate  information  on  production  and  cost  is  available. 

Since  this  type  of  contract  provides  fundamentally  for  a simple  exchange  of  a 
specified  sum  of  money  for  a specified  item,  it  is  the  easiest  and  least  costly  of  all  con- 
tract types  to  administer. 

Firm  fixed-price  contracts  can  be  used  for  research  or  development  work  when 
1 1 ) there  is  a proper  scope  of  work.  1 2)  the  contractor  assumes  the  risk  on  a cost  sharing 
basis,  or  (3)  the  contractor  recognizes  the  risk  and  is  willing  and  financially  able  to  take  it. 

FIXED  PRICE  WITH  ESCALATION 

The  fixed  price  contract  with  escalation  provides  for  the  upward  and  downward 
revision  of  the  proposed  price  upon  the  occurrence  of  certain  contingencies  which  are 
specifically  defined  in  the  contract. 

Price  escalation  provides  for  an  adjustment  of  the  contract  price  on  the  basis  of  i 
increases  or  decreases  from  an  agreed-upon  level  in  published  or  established  prices  ol  spe- 
cific items  or  in  price  levels  of  the  contract  end  item. 
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The  use  of  this  type  of  contract  is  appropriate  where  serious  doubt  exists  as  to 
the  stability  of  the  market  and  labor  condition  which  will  exist  during  an  extended  period 
of  production,  and  where  contingencies  which  would  otherwise  be  included  in  a firm 
fixed-price  contract  are  identifiable  and  can  be  covered  separately  by  escalation.  This 
type  of  contract  is  difficult  to  administer. 

Escalation  will  usually  be  restricted  to  industry-wide  contingencies  and  labor  and 
material  escalation  will  be  limited  to  contingencies  beyond  the  normal  control  of  the 
contractor.  This  type  of  contract  is  limited  to  long-term  production  contracts  for  standard 
items. 

Labor  and  material  escalation  provides  for  the  adjustment  of  the  contract  price 
on  the  basis  of  increases  or  decreases  from  agreed  standards  or  indices  in  wage  rates,  specific- 
material  costs,  or  both. 

FIXED-PRICE  INCENTIVE  CONTRACTS 

The  fixed-price  cost  incentive  contract  is  a fixed-price  type  contract  with  provision 
for  adjustment  of  profit  and  establishment  of  the  final  contract  price  by  a formula  based 
on  a relationship  which  final  negotiated  total  costs  bear  to  total  target  costs.  An  incentive 
contract  includes  a target  cost,  a target  profit,  a price  ceiling  (but  not  a profit  ceiling  or 
floor),  and  a formula  for  establishing  final  profit  and  price.  After  performance  ot  the 
contract,  the  final  price  is  negotiated  and  the  final  contract  price  is  then  established  in 
accordance  with  the  formula. 

Fixed-price  incentive  contracts  are  appropriate  when,  due  to  the  nature  of  the  work 
required,  neither  the  contractor  nor  the  government  has  the  confidence  to  negotiate  a firm 
fixed  price,  bu(  the  contractor  is  willing  to  take  the  risk  at  the  ceiling  price  established. 

COST-TYPE  CONTRACTS 

The  cost  reimbursement  type  contract  provides  for  payment  to  the  contractor  o( 
allowable  costs  incurred  in  the  performance  of  the  contract  to  the  extent  prescribed  in 
the  contract.  This  type  of  contract  establishes  an  estimate  of  total  cost  ( I ) lor  the  pur- 
pose of  obligation  of  funds,  and  (2)  to  establish  a ceiling  which  the  contractor  may  not 
exceed  except  at  his  own  risk  without  prior  approval  or  subsequent  rectification  by  the 
Contracting  Officer.  A cost-reimbursement-type  contract  is  considered  suitable  for  use 
only  when  the  uncertainties  involved  in  contract  performance  are  of  such  a magnitude 
that  cost  of  performance  cannot  be  estimated  with  sufficient  reasonableness  to  permit  use 
of  any  type  of  fixed-price  type  contract.  In  addition,  it  is  essential  that  the  contractor's 
cost  accounting  system  be  adequate  for  the  determination  of  costs  applicable  to  the 
contract.  The  contract  normally  provides  for  surveillance  by  the  buyer  to  ensure  that 
inefficient  and  wasteful  methods  are  not  used.  There  are  various  types  of  cost-type  con- 
tracts cost,  cost  sharing,  cost  plus  fixed  fee,  cost  plus  incentive  fee.  and  cost  plus  award 
fee. 
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COST  CONTRACT 


The  cost  contract  is  a cost-reimburscmcnt-type  contract  under  which  the  contractor 
receives  no  fee.  Under  this  type  of  contract,  the  government  agrees  to  reimburse  the  con- 
tractor for  allowable  costs  of  performance  as  governed  by  ASPR,  Section  XV,  and  specific 
terms  of  the  contract,  ll  is  used  for  research  and  development  work  with  educational 
institutions  and  other  nonprofit  institutions,  and  for  facilities  contracts. 


COST  S 11 A R 1 N O CO  N T R ACT 

The  cost-sharing  contract  is  one  under  which  the  contractor  receives  no  fee  and  is 
reimbursed  only  tor  an  agreed  portion  of  his  allowable  costs,  as  governed  by  ASPR. 
Section  XV.  The  cost-sharing  type  of  contract  recognizes  that  the  contractor  sometimes 
benefits  substantially  (apart  from  profit)  by  the  performance  of  a government  contract. 
This  is  particularly  true  in  the  field  of  development  work,  where  the  results  of  the  work 
performed  under  a government  contract  may  have  profitable  commercial  application.  Where 
the  prospect  of  commercial  application  can  be  foreseen  at  the  time  of  entering  into  the  con- 
tract. contracts  are  negotiated  which  provide  that  the  government  will  reimburse  the  contrac- 
tor for  only  a specified  percentage  of  its  costs.  The  unreimbursed  portion  of  the  cost  of  per 
formance  represents  the  contractor’s  contribution  to  what  is.  in  effect,  a joint  enterprise. 


COST-PLUS-A-FIXED-FEE  (CPFF)  CONTRACT 

The  cost-plus-a-fixed-fee  contract  is  a cost-reimbursement-type  contract  which  pro- 
vides for  the  payment  of  a fixed  fee  to  the  contractor.  In  addition,  the  contractor  is 
reimbursed  for  the  allowable  cost  of  performing  the  contract  as  governed  by  ASPR  Sec- 
tion XV,  and  the  terms  of  the  contract. 

Present  procurement  law  (10  U.S.C.  2306(d))  limits  the  fee  for  performing  a cost- 
plus-a-fixed-fee  contract  for  experimental,  developmental,  or  research  work  to  not  more 
than  15',;  of  the  estimated  cost  of  the  contract,  not  including  the  fee.  For  architectural 
or  engineering  services  for  a public  work  or  utility,  the  fee  is  limited  to  not  more  than 
6 7c  of  the  estimated  cost  of  that  work  or  project,  not  including  the  fee.  The  fee  for  per- 
forming any  other  cost-plus-a-fixed-fee  contract  may  not  be  more  than  IOC  of  the  esti- 
mated cost  of  the  contract,  not  including  the  fee. 

Because  the  cost-plus-a-fixed-fee  contract  obligates  the  government  to  reimburse 
the  contractor  for  the  allowable  cost  of  performing  the  contract  without  regard  to  the 
estimated  cost,  it  specifies  a maximum  amount  beyond  which  the  government  will  not  be 
obligated  to  reimburse  the  contractor.  The  maximum  may  be  more  or  less  than,  or  equal 
to,  the  estimated  cost.  The  contractor  agrees  to  use  his  best  efforts  to  complete  the  con- 
tract within  the  maximum  limitation,  but  has  no  obligation  for  further  performance  when, 
despite  his  best  efforts,  the  contract  is  not  fully  performed  at  the  time  the  maximum  has 
been  reached,  unless  the  Contracting  Officer  increases  the  maximum. 

Irrespective  of  whether  his  ucttuil  costs  are  greater  or  less  than  the  estimated  cost, 
the  contractor  receives  the  predetermined  fixed  fee.  If  the  scope  of  the  contract  work  is 
increased  or  decreased,  appropriate  increases  or  decreases  both  in  the  estimated  cost  and 
the  fixed  fee  are  negotiated. 


There  are  two  forms  of  cost-plus-a-fixed-fee  contracts: 

1.  The  Completion  form  is  one  that  describes  the  scope  of  work  to  be  done  as 

a clearly  defined  task  or  job  with  a definite  goal  or  target  expressed  and  with 
a specific  end  product  required.  This  type  is  used  for  the  development  ot 
hardware  or  where  a final  report  of  research  accomplishing  the  goal  or  target 
is  required. 

2.  The  Term  form  is  one  which  prescribes  the  scope  of  work  to  be  done  in 
general  terms  and  which  obligates  the  contractor  to  devote  a specified  level 
of  effort  for  a stated  period  of  time  for  the  conduct  of  research  and  devel- 
opment. This  type  of  contract  is  primarily  used  for  basic  research  and  the 
furnishing  of  level  of  effort  type  services. 

The  CPFF  contract  is  used  ( 1 ) for  the  performance  of  research,  preliminary 
exploration,  or  study  where  the  level  of  effort  required  is  unknown:  or  (2)  where  the 
contract  is  for  development  and  test  and  the  use  of  cost  plus  incentive  fee  is  not  practical. 


COST-PLUS-1NCF.NTIV E-FEE  (CPIF)  CONTRACT 

Under  this  type  of  contract,  the  government  and  the  contractor  agree  at  the  time 
of  negotiation  of  the  contract  upon  the  target  cost  of  performance.  The  target  fee  is 
then  determined  in  relation  to  the  target  cost.  Also  established  are  minimum  and  maxi- 
mum fees  and.  finally,  a fee  adjustment  formula. 

After  performance  of  a contract,  the  fee  payable  to  the  contractor  is  determined 
in  accordance  with  the  formula.  The  formula  provides,  within  limits,  for  an  increase  in 
fee  above  target  fee  when  the  total  allowable  costs  are  less  than  target  costs.  Conversely, 
it  provides  for  decreases  in  th-  fee  below  the  target  fee  when  the  total  allowable  costs 
exceed  the  target  costs. 

The  incentive-fee  contract  is  used  where  a cost-reimbursement-type  contract  is 
necessary  and  where  there  is  a probability  that  its  use  will  result  in  lower  costs  to  the 
government  than  other  forms  of  cost-reimbursement-type  contracts  through  cost-reduction 
incentive  to  the  contractor.  Maximum  fees  are  subject  to  the  same  percentage  limitations 
previously  mentioned  under  cost-plus-a-fixed-fee  contracts. 

The  CPIF  contract  is  suitable  for  use  primarily  for  development  and  test.  Where 
it  is  highly  probable  that  the  development  is  feasible  and  the  government  generally  has 
determined  its  desired  performance  objectives,  the  CPIF  contract  should  be  used  in  con- 
junction with  performance  incentives  in  the  development  ot  major  systems,  and  in  other 
development  programs  where  use  ol  the  cost  and  performance  approach  is  considered 
both  desirable  and  administratively  practical. 


COST-PLUS-AWARD-FEE  (CPAF)  CONTRACT 

The  cost-plus-award-fee  contract  is  a cost-reimbursement-type  contract  which 
provides  that  the  contractor’s  variable  fee  will  be  determined  subjectively  by  designated, 
high-level  government  personnel  on  the  basis  ol  periodic  alter-the-lact  evaluation  ol  the 
contractor's  performance.  The  CPAF  contract  has  a base  (in  some  cases,  the  base  fee 
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may  be  “zero”)  and  provision  tor  the  fee  to  be  adjusted  upward  on  the  basis  of  the  con- 
tractor's performance  evaluated  in  accordance  with  broad  criteria  set  forth  in  the  contract. 

The  award  fee  determination  is  the  subject  of  special  checks  and  balances  which  provide 
procedural  safeguards  protecting  the  contractor  from  arbitrary  or  capricious  evaluations, 
but  it  is  not  subject  to  the  Disputes  clause  procedure.  The  broad  criteria  against  which 
the  contractor’s  ultimate  performance  is  evaluated  include  performance  of  operations, 
technical  management,  business  management,  and  utilization  of  resources. 

CPAF  contracts  are  considered  appropriate  lor  support  services  generally  associated 
with  base  maintenance  and  operations  and  mission  support  contracts.  In  addition,  they 
are  used  for  contracts  for  the  operation  and  maintenance  of  computer  centers. 

TIME  AND  MATERIALS  CONTRACT 

The  time  and  materials  type  of  contract  provides  for  the  procurement  of  supplies 
or  services  on  the  basis  of  payment  for  direct  labor  hours  at  specified  fixed  hourly  rates 
(which  rates  include  direct  and  indirect  labor,  overhead,  and  profit)  and  materials  at  cost. 
Material  handling  costs  may  be  included  in  the  charge  for  “material  ar  cost,”  provided 
they  are  clearly  excluded  from  any  factor  of  the  charge  computed  against  direct  labor 
hours.  Under  this  type  of  contract,  a price  ceiling  is  established  which  the  contractor 
may  not  exceed,  except  at  his  own  risk. 

The  time  and  materials  contract  is  used  only  in  those  situations  where  it  is  not 
possible  at  the  time  of  placing  the  contract  to  estimate  the  extent  or  duration  of  the  work 
or  to  anticipate  costs  with  any  substantial  accuracy.  Its  use  is  restricted,  as  its  disadvan- 
tage is  obvious;  since  it  provides  for  payment  of  a fixed  price  per  applicable  unit  of  time, 
it  is  evident  that,  unless  the  rate  is  insufficient  to  cover  the  contractor's  costs,  the  total 
amount  of  profit  under  the  contract  is  increased  proportionately  as  the  number  of  hours 
are  increased:  Tor  this  reason,  the  time  and  materials  contract  is  not  preferred  and  is  used 
only  after  the  Contracting  Officer  has  determined  that  it  most  suitably  serves  the  requirement. 

This  type  of  contract  is  usually  used  for  (1)  repair,  maintenance,  or  overhaul  work, 
and  work  to  be  performed  in  emergency  situations;  (2)  engineering  design  and  preparation 
and  revision  of  drawings;  and  (3)  manufacture  of  dies.  jigs,  and  fixtures. 

LETTER  CONTRACTS 

A letter  contract  is  a written  preliminary  contractual  instrument  which  authorizes 
immediate  commencement  of  manufacture  of  supplies  or  performance  of  services,  includ- 
ing. but  not  limited  to.  preproduction  planning  and  the  procurement  of  necessarv  materials 
It  is  used  when  negotiation  of  a definitive  contract  in  sufficient  time  to  meet  the  procure 
ment  need  is  not  possible,  as.  for  example,  when  the  nature  of  the  work  involved  prevents 
the  preparation  of  definitive  requirements,  specifications,  or  cost  data. 
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APPENDIX  A:  GLOSSARY  OF  PROC  UREMENT  TERMS 


Term 

Administrative  Contracting 
Officer  ( ACO ) 


Armed  Services  Procurement 
Regulation  (ASPR) 

Bid 


Bidders  Conference 


Bidders  (Mailing)  List 
(Master  Bidders  List) 


Blanket  Purchase  Agreement 


“Boiler  Plate” 

Buy  American  Act 


Change  Order 


Commerce  Business  Daily 


Procurement  Usage 

A government  contracting  officer,  often  at  an  in- 
stallation other  than  the  one  which  made  the  con- 
tract. who  handles  the  business  administration  of 
the  contract.  Lor  the  larger  primes,  the  ACO  is 
commonly  resident  at  the  prime's  facility. 

The  basic  regulation  for  the  conduct  of  Defense 
Department  procurement.  Further  implemented 
by  Departmental  Procurement  Instruction. 

A prospective  contractor’s  (bidder’s)  reply  to  a 
formally  advertised  solicitation  document  (11  B) 
Needs  only  government  acceptance  to  constitute 
a binding  contract. 

In  formally  advertised  procurements,  a meeting 
of  prospective  bidders  arranged  by  the  contracting 
officer  during  the  solicitation  period  to  help  soli- 
cited firms  fully  understand  the  government's  re- 
quirement and  to  give  them  an  opportunity  to  ask 
questions.  (For  research  and  development  procure- 
ments. see  Presolicitation  Conference.) 

List  of  sources  maintained  by  the  procuring  otlice 
from  which  bids  (formal  advertising)  or  proposals 
or  quotations  (negotiation)  can  be  solicited. 

A negotiated  contractor  agreement  between  a con- 
tractor and  the  government  under  which  individual 
purchase  orders  not  exceeding  $2500  may  be  placed 
for  a specified  period  of  time  and  within  a stipu- 
lated  aggregate  amount. 

See  General  Provisions. 

Federal  statute  imposing  restiictions  on  placing 
contiacls  with  manufacturers  who  would  deliver 
items  not  substantially  produced  in  (lie  United 
States. 

Unilateral  direction  to  a contractor  to  modi  I \ a 
contractual  requirement  within  the  scope  ot  the 
contract,  pursuant  to  the  Changes  clause  contained 
in  the  contract.  See  also  Modification  and  Supple- 
mental Agreement 

See  Department  of  Commerce.  Commerce  Busi- 
ness  Daily . 
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Contract  Type- 


Basic  Agreement 


Cost-Reimbursement 

Contracts 


Fixed-Price  Contracts 


Indefinite  Delivery 


Fetter 


Normally,  a reference  to  the  pricing  terms  of  the 
agreement  between  a buyer  and  a seller,  but  may 
refer  to  the  special  nature  of  other  important  terms 
in  the  agreement.  Thus,  a contract  may  he  a "fixed 
price”  type.  Further,  a “letter  contract”  may  be 
either  a fixed-price  type  or  a cost-reimbursement 
type. 

A written  instrument  of  understanding  (not  a legally 
enforceable  contract  per  se)  between  a contractor(s) 
and  the  government.  Sets  forth  the  contract  clauses 
applicable  to  future  procurements  entered  into  be- 
tween the  parties  during  the  term  of  the  basic  agree- 
ment. Used  to  eliminate  extensive  and  costly  nego- 
tiation when  a substantial  number  of  separate  con- 
tracts may  be  entered  into  with  a contractor  over 
a definite  period  of  time. 

In  general,  a category  of  contracts  whose  use  is 
based  on  payment  by  the  government  to  a contractor 
of  allowable  costs  as  prescribed  by  the  contract. 
Normally  only  “best  efforts”  of  the  contractor  are 
involved.  Includes  (i)  cost,  (ii)  cost  sharing,  (iii) 
cost-plus-fixed-fee,  and  ( iv ) cost-plus-incentive-fee 
contracts. 

In  general,  a category  of  contracts  whose  use  is 
based  on  the  establishment  of  a firm  price  to  com- 
plete the  required  work.  Includes  (i)  firm  fixed- 
price.  (ii)  fixed-price  with  escalation,  (iii)  fixed- 
price  redeterminable.  and  (iv)  fixed-price  with  in- 
centive provisions  contracts. 

Used  when  the  precise  quantity  of  items  or  specific 
time  of  delivery  desired  is  not  known.  Usually  will 
specify  a maximum  and/or  minimum  quantity.  Such 
procurement  is  effected  via  (i)  a definite  quantity 
contract,  (ii)  a requirements  contract,  or  (iii)  an  in- 
definite quantity  contract.  May  be  either  negotiated 
or  formally  advertised. 

An  interim  type  of  contractual  agreement,  sometimes 
called  a “Letter  of  Intent,”  authorizing  the  com- 
mencement of  manufacture  of  supplies  or  perform- 
ance of  services.  Used  in  negotiated  procurements 
only  when  a definiti/ed  fixed-price  or  cost-reimburse- 
ment contract  cannot  be  written  until  a later  date. 
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Special  Purpose  Contracts 


fime  ami  Materials/Labor 
I lour  Contracts 


Cost  Overrun 


Data 


Defense  Contract  Administra- 
tion Service  (DCAS) 


Determination  and  f indings 
(D&F) 


Fxtras 


Formal  Advertising 


Hid 


In  general,  a category  of  contracts  designed  to  facili- 
tate a procurement  for  which  one  of  the  fixed-price 
or  cost-reimbursement-type  contracts  is  considered 
inappropriate.  Includes  (i)  basic  agreements,  (ii)  in- 
definite delivery  type  contracts,  (iii)  letter  contracts, 
and  (iv)  time  and  materials/labor  hour  contracts. 

Negotiated  contracts  based  on  specified  fixed  hourly 
rates  to  complete  a given  task.  Used  only  in  situa- 
tions where  it  is  not  possible  at  the  outset  to  esti- 
mate the  extent  or  duration  of  the  work  involved  or 
to  anticipate  cost  with  any  substantial  accuracy. 

I lie  amount  by  which  a contractor  exceeds  (i)  the 
estimated  cost  and/or  (ii)  the  final  limitation 
(ceiling)  of  his  contract. 

All  recorded  information  to  be  delivered  under  a 
contract.  “Technical  data”  excludes  management 
and  financial  data. 

An  agency,  under  direction  of  Director  of  DSA. 
created  as  result  of  Project  (SO  to  provide  unified 
contract  administration  services  to  Dol)  compo- 
nents and  NASA,  for  all  contract s except  those 
specifically  exempted. 

Written  justification  by  a contracting  officer  or 
higher  authority  for  (i)  entering  into  contracts  by 
negotiation,  (ii)  making  advance  payments  in  nego- 
tiated procurements,  and  tiii)  determining  the  type 
of  contract  to  use. 

Additions  to  items  being  procured,  or  any  quantity 
above  that  called  for  by  the  contract  (besides  allow- 
able variation  in  quantity),  or  any  combination  of 
these  two. 

I he  preferred  method  for  government  procurement 
of  supplies  and  services.  After  public  opening  of 
sealed  competitive  bids,  award  is  made  to  the  lowest 
responsive  and  responsible  bidder,  price  and  other 
factors  considered,  in  accordance  with  ASPR  Sec- 
tion II. 

A prospective  contractor's  (bidder's)  reply  to  the 
solicitation  form  used  for  formallv  advertised  pro- 
curements (I FID.  Needs  only  the  government's 
acceptance  for  award  to  be  made. 
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Bidders  Conference 


\ 

I 


Invitation  for  Bids  (IFB) 


Request  for  Technical 
Proposal  ( RTP) 

Responsive  and  Responsible 
Bidder 


Two-Step  Advertising 


General  Accounting  Office 
(GAO) 


General  Provisions 


G o ve  rn  m e n t-Fu  m ished 
Property  (GFP) 


Master  (Task  Order) 
Contract 


A conference  held  after  solicitation,  but  before 
bid  opening  of  all  those  suppliers  who  were  invited 
to  bid  on  the  procurement  to  discuss  details  of  the 
Invitation  for  Bids. 

The  solicitation  form  used  in  formally  advertised 
procurements  and  in  step  two  of  two-step  advertising 
(see  below). 

The  solicitation  form  used  to  request  proposals  in 
the  first  step  of  two-step  advertising. 

A bidder  is  responsive  if  his  bid  conforms  to  the 
requirements  of  the  1FB,  and  is  responsible  if  he 
has  the  capacity  and  facilities  to  produce  the  sup- 
plies or  render  the  services  being  procured. 

A method  of  procurement  authorized  by  ASPR 
Section  II,  Part  5.  Step  one  consists  in  requesting 
technical  proposals  from  prospective  contractors. 

Step  two  consists  in  inviting  bids  under  regular 
formal  advertising  procedures  from  those  linns  that 
submit  acceptable  proposals  under  step  one. 

An  agency  of  the  legislative  branch,  responsible 
solely  to  the  Congress,  which  functions  to  audit 
all  negotiated  government  contracts  and  investi- 
gate all  matters  relating  to  the  receipt,  disburse- 
ment. and  application  of  public  funds.  Determines 
whether  public  funds  are  expended  in  accordance 
with  appropriations. 

The  mandatory  (by  law  or  regulation)  clauses  lor 
a II  Dot)  contract s for  the  type  of  procurement 
involved  sometimes  called  “boiler  plate."  The 
clauses  devised  particularly  for  the  procurement 
are  called  the  Special  Provisions. 

Government-owned  property  furnished  to  a con- 
tractor for  the  performance  of  a contract.  Defined 
as  (i)  industrial  facilities,  (ii)  material,  tin)  special 
tooling,  (iv)  special  test  equipment.  ( v ) military 
property.  Also  designated  GF'M.  Government-Fur- 
nished Material,  and  GIT.  Government-Furnished 
Equipment. 

A type  of  agreement  describing  the  total  desired 
area  of  contractor  performance  and  breaking  this 
down  into  a number  of  broadly  defined  tasks  I'he 
contractor  is  obligated  to  perform  I ask  Orders  sub- 
sequently issued  by  the  government  under  the  terms 
and  conditions  in  the  Master  Contract 
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Any  formal  revision  of  the  terms  of  a contract, 
cither  within  or  outside  the  scope  ot  the  agreement. 
Includes  Change  Orders.  See  also  Supplemental 
Agreement. 

The  method  of  procurement  used  when  one  or 
more  of  the  basic  conditions  incident  to  formal  ad- 
vertising is  absent  and/or  when  there  is  justification 
under  one  or  more  of  the  seventeen  exceptions  pro- 
vided by  Title  10.  U.S.C.  2304(a).  See  Section  III. 
ASPR.  Appendix  I).  and  Negotiation  Authority, 
below. 

A prospective  contractor’s  response  to  the  solicita- 
tion form  (RFP/RFQ)  used  lor  a negotiated  pro- 
curement. 

A conference  between  government  and  industry 
concerning  technical  problems  or  other  areas  of 
importance  relating  to  the  procurement.  Precedes 
the  solicitation  of  prospective  contractors  for  the 
procurement. 

The  solicitation  form  used  for  negotiated  procure- 
ments when  the  government  reserves  the  right  to 
award  without  further  oral  or  written  negotiation. 
Only  the  acceptance  of  the  government  is  required 
for  award. 

Request  for  Quotation  (RFQ)  The  solicitation  form  used  in  negotiation  to  ob- 
tain price,  cost,  delivery,  and  other  information 
from  prospective  suppliers.  Used  when  award  will 
be  made  following  extensive  negotiation  with  the 
offeror.  Award  requires  bilateral  agreement  of 
both  the  contractor  and  the  government. 

Negotiation  Authority  Authority  to  negotiate  a contract  under  one  of  the 

seventeen  statutory  exceptions  granted  by  Congress, 
rather  than  to  formally  advertise  the  procurement. 
ASPR  Section  III  describes  circumstances  in  which 
this  authority  may  be  used  and  the  requirements 
for  its  approval. 

Offer  A prospective  contractor's  response  to  a solicita- 

tion document  in  a negotiated  procurement  The 
contractor  can  be  termed  the  “offeror.”  See  also 
RI  P and  RFQ. 

Option  A contractual  clause  permitting  an  increase  in  the 

quantity  of  supplies  beyond  that  originally  stipu 
lated  or  an  extension  in  the  tune  for  which  services 
on  a time  basis  may  be  required 


Modification 


Negotiation 


Offer/Proposal/Quotation 


Presol i c i t a t ion  Con fe re  n ce 


Request  for  Proposal  (RF'P) 


Preaward  Survey  (Facility 
Capability  Review  I CR) 


Preproposal  Conference 


Pre so  1 ic  i t a t io n Con  fere  ne e 


Price  and  Fee 
Ceiling  Price 


Fee 


Target  Price 


Procurement  Contracting 
Officer  (PCO) 


Procurement  Request  (PR) 


I 


Study  of  a prospective  contractor's  financial,  or- 
ganizational, and  operational  status  made  prior  to 
contract  award  to  determine  his  responsibility 
and  eligibility  for  government  procurement 

In  negotiated  procurements,  a meeting  held  with 
potential  contractors  a few  days  after  requests  for 
proposals  have  been  sent  out.  to  promote  uniform 
interpretation  of  work  statements  and  specifications 
by  all  prospective  contractors.  See  also  Bidders 
Conference. 

A meeting  held  with  potential  contractors  prior  to 
a formal  solicitation,  to  discuss  technical  and  other 
problems  connected  with  a proposed  procurement 
The  conference  is  also  used  to  elicit  the  interest  ot 
prospective  contractors  in  pursuing  the  task 


The  negotiated  monetary  limit  in  a fixed-price 
type  contract  to  the  amount  that  the  govern- 
ment is  obligated  to  pay.  Cost  incurred  beyond 
this  point  must  be  absorbed  by  the  contractor. 

An  amount,  in  addition  to  allowable  costs,  paid 
to  contractors  having  Cl’FF  or  CPU  contracts 
In  CPFF  contracts  the  fee  is  fixed  as  a percentage 
(stated  in  a dollar  amount)  of  the  initially  esti- 
mated cost  of  the  procurement.  In  CPU  contracts 
the  fee  is  expressed  in  maximum  and  minimum 
amounts,  along  with  a fee-adjustment  formula 
that  provides  the  incentive  for  a reduction  in  cost 
to  the  government.  Statutory  limitations  are  pre- 
scribed for  the  maximum  setting  of  fees. 

The  negotiated  estimate  of  price  in  a fixed-price 
undeterminable  or  incentive  contract  that  the 
government  expects  to  pay  for  supplies  procured 
under  the  contract. 

The  government  contracting  officer  directing  and 
administering  the  procurement  through  the  award 
of  the  contract  and  the  signing  ol  the  actual  con- 
tractual documents.  Administration  of  the  con- 
tract after  award  may  be  delegated  to  an  A(  O. 
as  described  above. 

1 

Document  which  describes  the  required  supplies 
or  services  so  that  a procurement  can  be  initiated 
Some  procuring  activities  actually  refer  to  the  docu- 
ment by  this  title,  others  use  different  titles  such 
as  Procurement  Directive,  and  so  forth 


. 
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Purchase  Order 


Qualified  Products  List  (QPL) 


Supplemental  Agreement 


Task  Order 
Termination 


A contractual  procurement  document  used  pri- 
marily to  procure  supplies  and  nonpersonal  serv- 
ices when  the  aggregate  amount  involved  in  any 
one  transaction  is  relatively  small  (lor  example, 
not  exceeding  S 2500 ).  1)1)  Form  1 155. 

A list  of  products  which  are  pretested  in  ad- 
vance of  actual  procurement  to  determine  which 
suppliers  can  comply  properly  with  specifica- 
tion requirements.  This  is  most  usually  done 
because  of  the  length  of  time  required  for  test 
and  evaluation. 

Bilateral  written  amendment  to  a contract  by 
which  the  government  and  the  contractor  settle 
price  and/or  performance  adjustments  to  the 
basic  contract.  See  also  Change  Order  and  Modi- 
fication. 

See  Master  Contract. 

The  canceling  of  all  or  a part  of  a prime  or  sub- 
contract prior  to  its  completion  through  per- 
formance. 
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APPENDIX  B:  INSTRUCTIONS  FOR  COMPLETION  OF  1)1)  FORM  1423. 
"CONTRACT  DATA  REQUIREMENTS  LIST” 


1 . Introduction : 

a.  When  a Contract  Data  Requirements  List  (C'DRL).  is  made  part  of  a contract, 
it  will  be  the  sole  contractual  document  listing  all  data  to  be  delivered  under 
the  contract. 

b.  Blocks  17  through  26  are  detachable.  (See  paragraph  3.q.) 

c.  Blocks  1.  2.  4,  5.  6,  7,  8.  10.  12.  13,  14.  15.  25.  and  26  must  be  completed 
when  applicable.  Completion  of  fields  for  other  data  elements  is  optional. 

d.  A maximum  of  four  separate  line  items  of  data  may  be  entered  on  each  form. 

e.  DD  Form  1423  may  be  locally  reproduced  on  8 x 13-inch  paper. 

f.  Special  instructions  pertaining  to  use  of  the  DD  Form  1423  in  solicitations  are 
provided  in  ASPR  16-815. 

2.  Cieneral  CDRL  Information: 

a.  Attachment  Number.  Leave  blank.  The  CDRL  is  an  exhibit  to  the  contract 
and  not  an  attachment  to  any  other  exhibit. 

b.  To  Exhibit.  Enter  exhibit  identifier  in  accordance  with  ASPR  20-305.2.  The 
composite  CDRL  will  be  one  exhibit  to  the  data  line  item  on  the  contract. 

c.  To  Contract/PR.  Enter  the  actual  contract  number,  if  known;  if  unknown, 
enter  the  purchase  request  number  or  other  appropriate  designator. 

d.  Category.  If  categories  of  data  are  used,  such  as  logistics  or  test,  the  CDRL 
may  be  divided  using  these  categories.  A new  DD  Form  1423  will  normally 
be  started  for  each  category.  If.  however,  the  CDRL  contains  a few  items 
and  numerous  categories,  this  line  should  read  “NA.” 

e.  System/ltcm.  Enter  the  system,  item,  project  designator,  or  name. 

f.  Contractor.  Place  contractor’s  name  or  identification  code  in  this  field  when 
appropriate.  Following  the  name,  a slash  (/)  and  the  contractor's  Federal 
supply  manufacturer’s  code  (FSMO  may  be  inserted.  This  code  may  be  ob- 
tained from  DoD  Handbook  11-4. 

g.  Prepared  By/Date.  Enter  the  name,  title,  office  code,  and  signature  of  the 
person  responsible  for  preparing  the  CDRL  and  the  preparation  date. 

h.  Approved  By/Date.  Enter  the  name  and  signature  of  the  chairman  of  the 
DRRB  for  those  contracts  estimated  total  value  in  excess  of  SI. 000. 000  or 
the  program  manager  (or  his  designated  alternate)  for  those  contracts  under 
SI. 000.000  and  the  approval  date. 

i.  Contract  Value.  Insert  the  price  or  estimated  total  cost  of  the  contract,  pur- 
chase request,  etc. 

NOLI  Items  2(b)  and  (0  must  be  completed  on  the  first  page  of  the  CDRL  only. 

Items  2(c).  (d).  and  <e(  should  be  completed  on  all  pages  for  ease  of  identi- 
fication. Items  2(g),  (h)  and  (i)  must  be  completed  on  the  last  page  of  the 
CDRL  only  Item  2li)  need  not  be  completed  until  negotiations  are  com- 
pleted. 


3.  Detailed  CDRL  Information: 

a.  Block  1,  Sequence  Number.  Number  the  data  items  in  accordance  with 
ASPR  20-306.2. 

b.  Block  2,  Title  or  Description  of  Data.  Filter  the  official  title  or  a brief  de- 
scription of  the  data.  If  the  title  is  selected  from  the  DODADL.  it  must  be 
identical  to  the  DODADL  title.  Block  3 is  to  be  used  for  further  identifica- 
tion if  required. 

c.  Block  3,  Subtitle  of  Data.  If  the  title  requires  further  identification,  enter  a 
subtitle. 

d . Block  4.  Authority  (Data  Item  Description  Number) . 

( I ) Standard  Data  Items.  Insert  selected  Data  Item  Description  numbers  from 
DODADL.  If  the  Data  Item  Description  (1)1)  Form  1664)  has  a dash  (-) 
letter  following  it  (indicating  a revision)  include  this  letter  in  the  basic 
number  (for  example.  “DI-V-1  10-A").  If  more  than  one  data  item  is  used 
to  construct  a specific  data  requirement,  each  data  item  will  be  separately 
listed  on  the  CDRL,  and  block  16  may  be  used  to  indicate  the  relation- 
ship (for  example.  "Combine  with  contract  data  item (for  sub- 

mission"; or  "Data  prepared  in  accordance  with  Data  Item  Description 
DI-F-xxxx). 

(2)  Unique  ("U”)  Data  Item  Descriptions.  If  no  Data  Item  Description  m the 
ADL  will  meet  the  requirements  of  the  program  prepuie  a “11”  Dll)  on 
DD  Form  1664  and  attach  it  to  DD  Form  1423  The  appropriate  item 
category  will  be  entered,  and  followed  by  a local  control  number  (foi  ex- 
ample. "UDi-A-201 23").  Do  not  use  control  numbers  identical  to  existing 
standard  data  items.  “U”  Data  Item  Descriptions  must  be  forwarded  to 
the  Command  data  management  office  for  approval. 

(3)  Technical  Publications; 

(a)  For  large  system  procurements,  list  technical  publications  as  a group 
“Technical  Publications”  on  the  CDRL  and  cite  the  Data  Item  De- 
scription which  refers  to  the  Technical  Manual  Contract  Requirements 
when  the  number  of  publications  (and  resulting  changes)  would  cause 
the  CDRL  to  become  unwieldy  or  when  required  publications  cannot 
be  individually  identified.  For  this  system,  use  the  contract  work 
statement  (or  attachments)  as  the  control  document  for  technical  pub- 
lications. 

(b)  For  small  systems  and  equipment  procurements,  publications  may  be 
listed  individually  on  the  CDRL. 

(cl  The  decision  to  list  individual  technical  publications  on  the  CDRl  or 
a group  listing  will  be  a coordinated  decision  of  the  requiring  organi- 
zation and  the  data  management  officer. 

e.  Block  5.  Contract  Reference.  I liter  the  specific  paragraph  number  of  the  con 
tract  or  purchase  request  and  or  its  exhibits  which  will  assist  in  identify  ing  the 
data  requirement  authorized  by  block  4.  Ibis  is  particularly  important  when 
the  data  are  a by-product  of  contract  tasks 
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f.  Block  6,  Technical  Office.  Enter  the  office  responsible  for  advising  on  the 
technical  adequacy  of  the  data.  This  may  be  the  accepting,  requiring,  using, 
or  inspecting  offices  depending  on  the  type  of  data  and  decisions  made  rela- 
tive to  quality  assurance  responsibilities.  The  designated  accepting  office 
(block  7)  will  consult,  if  required,  with  the  office  listed  in  block  6 in  perform- 
ing the  acceptance  function. 

g.  Block  7,  DD  250  Requirement.  The  program  manager  will  designate  the  loca- 
tion (contractor’s  facility,  or  destination)  for  performance  of  Government 
quality  assurance  and  acceptance.  This  is  accomplished  by  entering  the  appli- 
cable code  for  quality  assurance  and  acceptance.  The  activity  to  perform  the 
destination  acceptance  task  will  be  entered  in  block  14  as  the  first  addressee. 
("Same  as  block  6”  if  appropriate.)  The  1)D  Form  250,  “Material  Inspection 
and  Receiving  Report,”  will  be  indicated  as  a requirement  for  engineering 
drawings,  specification,  and  related  data.  It  is  optional  for  other  types  of 
data. 

h.  Block  8,  Approval  Code.  Use  this  block  to  identify  requirements  for  advance 
written  approval  and  the  requirement  to  use  appropriate  distribution  state- 
ments prior  to  publication  and  distribution  of  the  final  document.  The  follow- 
ing codes  should  be  inserted  as  appropriate: 

Code  When  Used 

"A”  The  data  item  is  critical  and  requires  written  approval  (as  for  test  plans). 

Most  of  these  data  require  submission  of  a preliminary  draft  prior  to 
publication  of  a final  document.  Scope  and  Depth  of  approval  will  be 
explained  in  block  1 6. 

"D”  A distribution  statement  is  known  to  be  required.  This  must  either  be 

(I)  cited  in  block  16.  (2)  specified  in  the  DID.  or  (3)  obtained  from  the 
program  office. 

"AD”  Both  approval  and  distribution  statements  are  required 

"N"  Not  applicable  to  these  data. 

“AN”  Approval  required. 

BLANK  Approval  is  not  required. 

i Block  9.  Input  to  1AC.  If  data  are  dependent  upon  the  integrated  result  of 
specific  inputs  from  other  participating  contractors,  place  an  "X"  in  this  block. 
In  all  other  cases,  leave  the  block  blank. 

j.  Block  10,  Frequency.  Enter  the  appropriate  frequency  code  as  indicated  and 
use  block  13  for  further  explanation.  If  data  are  of  recurring  type,  they  will 
be  submitted  at  the  end  of  the  report  period  established  in  this  field  unless 
otherwise  indicated  in  the  DD  Form  1664  “Preparation  Instructions"  block 
or  in  block  12  or  13.  Use  of  the  term  “ASRFQ"  (as  required)  must  be  further 
amplified  in  either  block  13  or  block  16  to  define  the  ASREQ  conditions. 

This  is  not  always  self-evident  and  loss  of  control  could  result  if  it  is  left  am- 
biguous. 
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DAILY 

Daily 

ANNLY 

Annually 

WlkLY 

Weekly 

SEMI  A 

Each  6 months 

BI-WE 

Each  2 weeks 

OTIME 

One  time 

MTHLY 

Monthly 

ONE/R 

One  time  & revisions 

BI-MO 

Each  2 months 

R/ASR 

Revisions  as  required 

QRTLY 

Quarterly 

ASREQ 

As  required 

2/MTH 

Semi  Monthly 

CP/RQ 

Change  pages  as  required 

2TIMI 

Two  separate  submittals 

DEDEL 

Deferred  delivery 

XTIME 

X Separate  submittals 

ONE/P 

One-time  preliminary  draft 

k.  Block  1 1 , As  Of  Date.  If  the  data  are  submitted  only  once,  enter  the  “as  of" 
date  (cutoff  data)  as  follows:  year/month/day  (“1969  March  5”  rather  than 
"5/3/69").  If  date  is  constrained  by  a specific  event  or  milestone,  enter  this 
constraint.  If  the  data  are  of  a recurring  type,  enter  the  number  of  days 
prior  to  the  end  of  the  report  period;  for  example,  "15"  would  place  the 

"as  of"  date  for  this  report  at  15  days  before  the  end  of  each  month,  quart  r. 
or  year  depending  upon  the  frequency  established  in  block  10:  “O”  would  place 
the  "as  of”  date  at  the  end  of  each  month,  quarter,  etc.  (Blocks  13  or  16  may 
be  used  for  further  explanation.) 

l.  Block  12.  Dale  for  hirst  Submission,  baiter  initial  data  submission  date  as 
follows:  year/month/day  (for  example,  “1969  March  5").  If  data  have  already 
been  submitted  and  will  be  resubmitted,  enter  date  of  next  submission.  If  date 
is  constrained  by  a specific  event  or  milestone,  enter  this  constraint.  If  con- 
tract start  date  is  not  known,  indicate  the  numbers  of  days  after  contract  start 
that  the  data  are  due  (example:  "30  DAC”).  If  this  date  is  not  known  or  re- 
quires further  clarification,  use  block  13.  If  deferred  delivery  is  involved,  in- 
sert "DEDEL".  ( NOTb : Do  not  insert  classified  dates.) 

m Block  13,  Date  of  Subsequent  Submissions/Evcnt  Identification.  If  data  are 
submitted  more  than  once,  enter  the  dates  of  subsequent  submissions.  If  date 
is  constrained  by  a specific  event  or  milestone,  enter  this  constraint;  for  ex- 
ample. "NLT  15  days  before  start  of  production,  45  days  before  launch.”  If 
this  information  classifies  the  list,  leave  blank. 

n.  Block  14.  Distribution  and  Addressees.  Enter  the  addressees  and  the  number 

of  copies  (regular  or  reproducible)  each  is  to  receive  (for  example,  “DDC-20/0”). 
The  first  addressee  should  be  the  acceptance  activity  for  the  data  if  acceptance 
is  to  be  accomplished  at  destination.  Office  codes,  contractor  initials.  Dol) 
Handbook  11-4  numbers.  Navy  Standard  Distribution  List,  and  command  ini- 
tials may  be  used.  (Attach  a list  explaining  these  codes  to  the  CDRL.)  If  repro- 
ducible copies  are  required,  explain  in  this  field  or  use  block  16:  for  example, 
"offset  master",  "vellum",  "negative".  If  deferred  delivery  is  desired  under 
ASI’R  9-202.2(0.  indicate  by  placing  the  word  “deferred”  or  similar  remarks 
in  this  field. 

o.  Block  I 5.  Total,  (inter  the  total  number  of  regular  and/or  type  of  reproducible 
copies.  This  may  be  obtained  by  adding  all  of  the  insertions  in  block  14 

p Block  16,  Remarks,  (inter  in  this  field  all  pertinent  data  item  information  not 
specified  elsewhere  on  the  form  and  any  required  amplification  of  other  block 
inputs.  Ellis  may  include  data  item  modifications,  (i.c.,  tailoring  data  item 
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to  specific  procurement),  special  packaging  and  delivery  information,  ampli- 
fication of  “deferred”  status,  etc.  If  additional  space  is  required,  use  block  16 
of  the  next  line  or  an  attached  sheet.  (NOTE:  In  lieu  of  using  just  block  16 
of  the  next  line,  blanking  out  blocks  1 through  15  to  provide  more  space  is 
also  acceptable. 

q.  Block  17  through  26.  These  blocks  are  not  contractual  information  but  are 
used  in  negotiating  and  preparing  the  contract.  They  are  detachable  from  OD 
Form  1423  and  will  not  be  made  a part  of  the  contract.  However,  the  ori- 
ginal 00  Form  1423  as  submitted  by  the  contractor  will  be  retained  in  the 
PCO  contract  tile. 

r.  Block  17.  Event/CFl  Number.  Insert  either  one  or  both  of  the  following: 

(1)  Event  Number.  This  is  the  identification  number  used  in  management 
systems  such  as  PERT. 

(2)  CFJ  Number.  This  is  the  configuration  item  number  used  in  configura- 
tion management  systems. 

s.  Block  18.  OMB  Approval/Form  Number.  If  the  data  equire  OMB  approval, 
enter  the  “Public  reports  approval  number”  which  has  been  assigned.  Follow- 
ing this,  enter  the  designated  Government  form  number.  This  information  is 
not  required  if  it  is  already  contained  in  the  DD  Form  1664. 

t.  Blocks  19-22.  This  information  must  be  completed  only  when  an  automatic 
data  distribution  system  is  used. 

u.  Blocks  23-26.  These  blocks  are  completed  by  the  contractor  in  accordance 
with  the  instructions  on  the  back  of  the  form,  see  ASPR  F200.1423. 
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APPENDIX  C:  STATEMENT  OF  WORK  REQUIREMENTS 


Statement  of  work  writers  should  ensure  that  objeetives  and  performance  require- 
ments in  the  SOW  are  complete  and  clearly  written  in  conventional  language  to  the  extent 
practical.  Complete  elimination  of  technical  language  is  not  required,  but  it  should  be  re- 
duced to  essentials  required  to  describe  the  task.  The  person  responsible  lor  preparing  the 
SOW  must  keep  in  mind  that  excess  technical  language  or  technical  constraints  can  affect 
the  procurement  beyond  simply  stating  or  directing  the  contract  effort.  It  may  also  affect 
the  number  of  good  sources  willing  and  able  to  respond. 


BRAND  NAME  OR  EQUAL 

Occasionally,  a requirement  will  be  of  such  nature  that  it  can  be  met  by  one  of  sev- 
eral commercial  products.  When  this  situation  exists,  it  is  frequently  possible  to  make  the 
procurement  on  a “Brand  Name  or  Equal”  basis.  Brand  name  does  not  require  a full  state- 
ment of  work,  but  does  oblige  the  requiring  activity  to  specify  all  the  technical  characteris- 
tics which  are  necessary  to  fulfill  the  requirement.  These  characteristics  become  the  speci- 
fication against  which  “equal”  products  can  be  measured.  A sample  SOW  for  a Brand  Name 
or  Equal  procurement  appears  at  the  end  of  this  appendix. 


SOW  CHECKLIST 

The  following  checklist  for  work  statements  provides  some  of  the  considerations 
which  the  writer  must  hear  in  mind: 

• Is  the  SOW  sufficiently  specific  to  permit  the  writer  and  the  contractor  to  make  a 
list  of  manpower  and  resources  needed  to  accomplish  it? 

• Are  specific  duties  of  the  contractor  stated  in  such  a way  that  he  knows  what  is 
required  and  the  technical  representative  who  signs  the  acceptance  report  can  tell 
whether  the  contractor  complied? 

• Are  sentences  written  so  that  there  is  no  question  as  to  whether  the  contractor  is 
to  be  obligated  (that  is.  “the  contractor  shall  do  this  work,”  not  “this  work  will 
be  required”)? 

• Is  the  proper  reference  document  shown?  Is  it  really  pertinent  to  the  task  ’ l ully 
or  partially?  Is  it  properly  cited? 

• Have  the  elements  of  quality  assurance  been  fully  considered  for  the  total  life  of 
the  procurement  requirement? 

• Are  any- military  specificat’  ns  or  exhibits  applicable?  In  whole  or  in  part'.’  It  so. 
are  they  properly  cited  ’ I Use  the  latest  available  revision  or  issue  ol  each 
document. ) 

• Is  general  information  separated  from  direction  so  that  background  information, 
suggested  procedures,  and  the  like  are  clearly  distinguishable  from  contractor 
responsibilities? 

• Is  there  a date  for  each  thing  the  contractor  is  to  do  or  deliver'.’  If  elapsed  tune  is 
used,  does  it  specify  calendar  days  or  work  days? 
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• Have  the  headings  been  cheeked  for  format  and  grammatical  usage'.’  Are  subhead- 
ings comparable'.’  Is  the  text  compatible  with  the  title?  Is  a multidecimal  num- 
bering system  used? 

• Have  extraneous  material  and  cross  references  to  contract  clauses  and  gcneul 
provisions  been  expunged? 

• Are  task  line  item  and  end  item  procurement  provisions  mutually  discrete  with 
regard  to  development  and  test  vs  production  activities? 

• Have  all  requirements  been  reviewed  to  ensure  compatibility  with  the  data  re- 
quirements specified  on  DD  Form  1423? 

• Have  all  extraneous  data  requirements  been  eliminated? 

• Are  all  obligations  of  the  government  carefully  delineated?  (If  government  fur- 
nished equipment  is  to  be  provided,  the  nature,  condition,  and  availability  of  the 
equipment  should  be  stated.  If  approval  actions  are  to  be  made  by  the  govern- 
ment. provide  for  a time  limit.  Remember  that  any  provision  which  takes  control 
of  the  work  away  from  the  contractor,  even  temporarily,  must  be  covered  by  a 
contingency  reserve  if  the  contractor  is  to  protect  himself.) 

• Have  all  loopholes  been  closed?  (Contractors  and  inspectors  go  by  the  letter  ol 
the  SOW.  The  contractor  may  refuse  to  do  something  that  is  only  referred  to. 
desired,  or  described  as  a goal.) 

• Is  the  requirement  completely  described?  (To  be  legal  and  binding,  an  agreement 
must  be  complete.  Not  only  for  reasons  of  legality . but  for  every  practical  appli- 
cation. it  is  necessary  that  the  details  be  complete.  Specify  when  and  where  as 
well  as  what.) 

• Since  they  generally  result  either  in  an  expensive  disagreement  or  in  a windfall  to 
the  contractor,  have  all  “catchall”  statements  been  eliminated? 

• Does  the  SOW  sole  source  the  work?  (The  SOW  specifies  a requirement  of  the 
government  and  is  supposedly  impartial  concerning  who  can  do  it.  In  keeping 
with  this  philosophy,  the  work  statement  itself  should  contain  no  reference  to 
sources  or  proprietary  talent.) 

• Is  the  requirement  overspecified?  (The  ideal  situation  is  to  specify  results  re- 
quired and  let  the  winning  contractor  find  the  best  method  ol  getting  there.) 

• Has  the  work  been  organized  into  tasks'.’  ( These  are  helpful  in  evaluating  and 
during  performance  may  be  used  for  control.) 

• Have  all  points  of  control,  where  needed,  been  included  ( for  example,  submission 
of  designs  for  approval )'.’ 

• Does  the  SOW  include  only  such  reports  and  documentation  as  are  really  needed 
for  control;  documentation  of  technical  results;  and  follow-on  procurements? 


I ORMAT  Ol  THI  STATEMENT  OF  WORK 

The  importance  of  work  statements  is  such  that  consistency  of  format  will  make 
them  much  easier  to  review  and  understand  I he  general  format  and  headings  lor  SOW  s are 
as  follows: 

1.0  Introduction  (Objective) 
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2.0  Scope 

3 0 General  background  (information,  constraints,  and  reference  documents) 

4.0  task  Technical  Requirements 

5.0  Reports.  Data,  and  other  Deliverables 
b.O  Special  Considerations 

STATEMENT  OF  WORK  CONTENT 

The  content  of  the  SOW  is  described  by  paragraph  below. 

1.0  Introduction  (Objective).  The  introduction  is  intended  to  give  a brief  overview  of 
the  specialty  area  and  leads  up  to  why  this  particular  new  program  is  being  pursued.  The 
overall  requirement  which  needs  fulfillment,  the  present  difficulties  or  deficiencies  which  do 
not  permit  the  requirement  to  be  met.  and  the  determinations  which  must  be  made  to  solve 
the  problems  should  be  outlined  briefly,  in  fully  understandable  terms.  Quite  often  an 
understanding  of  the  value  of  the  ' •vhnieal  objective  can  be  reinforced  by  inclusion  ol  an 
explanation  of  the  payoff  that  this  technical  objective  will  have  to  future  naval  systems 
capability.  In  framing  the  objective,  think  clearly  on  how  the  results  will  be  used.  The 
stated  objective  should  be  consistent  with  the  funds  planned  and  or  with  the  minimum 
requirements. 

2.0  Scope.  This  section  provides  an  overall  picture  of  the  desired  work  program  in 
concise  form.  The  scope  will  outline  the  various  phases  of  the  program  and  tie  down  the 
overall  limits  of  the  program  in  terms  ot  specific  technical  objectives,  time,  and  any  special 
provisions  or  limitations.  It  must  be  consistent  with  the  detailed  requirements.  I his  section 
should  also  describe  in  a clean-cut  statement  the  end  result  desired  or  what  the  “product"  ol 
the  effort  should  be.  Don’t  overextend  the  magnitude  expected,  or  an  overrun  may  be  the 
result. 

3.0  General  Background.  Include  any  background  information,  explanations,  or  con- 
straints which  are  necessary  in  order  to  understand  the  requirements.  Discuss  how  the 
procurement  arose:  indicate  its  relationship  to  previous,  concurrent,  and  future  operations, 
including  the  threat  analysis:  and  relate  details  which  reveal  its  purpose  and  significance. 
Statements  on  the  importance  of  the  new  work  may  also  be  included  I echniques  w Inch 
have  previously  been  tried  and  found  ineltective  should  be  included.  Frequently  it  is  best  to 
leave  the  writing  of  the  background  to  the  last.  The  listing  of  applicable  technical  reports 
resulting  from  the  Dl)(  bibliography  search  should  be  entered  here.  Any  such  listing  m this 
paragraph  is  for  information  only  and  is  not  contractually  obligatory  . All  contractually 
applicable  documents  must  be  cited  either  in  the  text  of  the  appropriate  task  or  in  a sepa- 
rate paragraph  entitled  "Applicable  Documents  I is  t . II  there  are  no  ap|  livable  repot  Is.  a 
comment  to  this  effect  should  be  made  on  the  supplemental  sheet  to  the  purchase  request 
but  not  included  in  the  SOW. 

4.0  Technical  Requirements  Tasks 

4.1  Areas  of  Consideration.  This  paragraph  should  define  the  work  to  be  accomplished 
and  indicate  the  main  steps  and  actions  which  are  required  ot  the  contractor  to  conduct  the 
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program  properly.  These  main  steps  constitute  the  work  phases  (recommended  approach). 

I he  technical  leadership  prov  ided  by  the  government  in  planning  and  establishing  the  con- 
tractual program  appears  here.  It  should  not  retied  an  attitude  that  this  is  the  only  ap- 
proach to  the  problem.  It  should  state  that  this  is  a suggested  method  but  new  or  unique 
ideas  supported  by  available  data  are  acceptable  and  encouraged.  This  paragraph  also  gives 
known  specific  phenomena  methods  which  could  contribute  to  a solution,  possible  correla- 
tion with  existing  knowledge,  operational  and  installation  environments  anticipated  for  the 
ultimate  operational  equipment,  and  other  factors,  including  all  available  foreign  technology 
information,  such  as  would  tend  to  assure  that  the  bidder  contractor  would  conduct  a fully 
effective  program. 

4.2  If  the  work  encompasses  several  areas  or  lends  itself  to  division  into  tasks,  this 
should  be  indicated.  The  essential  procedures  (that  is.  theoretical  analyses,  design,  fabrica- 
tion. checkout,  tests,  verification,  formulation  of  final  recommendations,  etc  i.  vv  it  h limits 
on  each,  constitute  the  bulk  of  this  paragraph.  In  some  cases,  the  engineer  may  wish  to 
indicate  the  percent  of  the  total  effort  each  phase  is  to  receive.  It  there  are  existing  specifi- 
cations with  paragraphs  that  define  what  you  want  to  have  the  contractor  do  in  terms  of 
tests,  etc.  use  them  or  incorporate  by  reference,  as  appropriate,  rather  than  compose  original 
paragraphs.  Specify  those  considerations  which  may  guide  the  contractor  in  Ins  analysis, 
design,  or  experimentation  on  the  designated  problem.  These  should  include  operational 
characteristics  (if  any ) or  other  factors  the  contractor  is  expected  to  consider  m performing 
under  the  contract.  Definitions  may  also  be  included  or  can  be  identified  in  a separate 
section. 

4..'  Itc  sure  that  limits  of  environment,  test  durations,  combustion  pressures,  data  re- 
cording. expansion  ratio,  mixture  ratio,  range  of  particle  si/e.  etc.  are  specified.  < rite ri a 
governing  the  number  of  designs,  types  of  propellants,  performance,  hardware  si/e.  numhei 
of  tests,  etc,  and  constraints  such  as  budget,  environmental,  produeibility  and  risk  levels 
should  be  included  in  the  definiti/ation  of  the  work  to  be  done  by  the  contractor.  I Ins  may 
be  better  set  forth  in  a detail  specification. 

4.4  Commit  yourself.  When  the  burden  of  definition  must  be  placed  on  the  olteror. 
clearly  impose  the  requirement  in  such  a manner  that  he  understands  that  lie  must  provide 
this  definition  in  the  bid  I if  this  is  what  is  wanted ) or  later  on  in  the  contractual  program  t it 
this  is  the  intent  i.  Any  specific  limitation  such  as  “not  desired”  or  "previously  tried"  tech- 
niques should  be  stated.  If  there  is  a primary  area  with  a secondary  contributing  or  limiting 
area,  these  should  be  defined.  Experimental  or  installation  environments  (known  or  antici- 
pated t.  scientific  or  technical  personnel,  or  other  resources  should  be  indicated.  When  the 
bidder  prov ides  definition  or  plans,  it  should  be  stipulated  that  these  are  subject  to  Navy 
approval. 

4.5  \ description  should  be  given  of  any  end  item  that  is  the  subject  ol  development.  Ii 
will  firmly  and  cl  'arly  define  the  required  work  for  such  tasks  as  those  listed  below 

a.  Review  of  curren1  literature  to  establish  a basis  tor  further  research,  analysis, 
inv  es|  igation.  or  experimentation. 

b.  Search  for  new  ideas  through  investigation  ot  van  > u s phenomena. 

c.  Paper  or  theoretical  atialy  sis  ol  ideas  in  relation  to  requirements,  ultimate  use. 
and  trade-off  capabilities 
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ci.  Computational  analysis  and  tbrinulation  of  mathematical  model. 

e.  1 experimentation  to  evolve  methods  of  instrumentation. 

f.  Derivation  of  a basic  equipment  design  or  experimental  assemblies. 

g.  Test  and  evaluation. 

4.(i  If  the  state  of  the  art  is  such  that  one  or  more  specific  methods  of  approach  to  the 
solution  are  to  be  followed,  this  section  should  indicate  the  desired  approach.  Il  no  specific 
approach  is  primarily  warranted  and  one  will  be  determined  on  the  basis  of  the  selected 
contractor's  technical  proposal,  this  need  not  be  mentioned,  and  this  section  should  include 
a statement  of  criteria  on  which  a choice  of  alternative  approaches  will  be  based. 

4.7  Scientific  and  Technical  Information.  Insert  the  following,  if  applicable:  "The 
contractor  shall  search  the  existing  sources  of  scientific  and  technical  information  to  deter- 
mine the  current  state  of  the  art  to  avoid  duplication  of  effort  and  conserve  scientific  and 
technical  resources."  Fnsure  that  all  generated  scientific  and  technical  information  that  has 
significant  value  to  the  pertinent  scientific  and  technical  communities  is  furnished  to  the 
Defense  Documentation  Center  ( DDO. 

5.0  Reports.  Data,  and  Other  Deliverables.  Contract  data  or  reporting  requirements 
should  not  be  duplicated  in  the  SOW.  DD  Form  1425  is  the  medium  for  establishing  the 
requirements.  The  SOW  may  refer  to  the  DD  Form  1425  incorporated  in  the  contract  bv 
reference  or  even  to  any  particular  data  item  for  clarifying  a requirement.  If  deliverable 
hardware  is  required,  it  should  also  be  listed  in  this  section  as  a separate  paragraph. 

0.0  Special  Considerations.  A paragraph  outlining  any  special  interrelationships  between 
the  contractors  for  use  of  government-furnished  or  loaned  property,  for  example,  max  be 
devised  and  added  to  the  SOW  as  paragraph  6.0.  Any  other  specific  directions  relative  to 
technical  work  (not  administrative  matters)  for  the  contractor  to  follow  should  be  included 
here.  This  paragraph  might  also  provide  instructions  to  the  contractor  relative  to  the  possi- 
ble utilization  of  government  expertise. 

WORK  PACKAGE  S 

An  integral  aspect  of  both  understanding  and  developing  a statement  of  work  is  the 
ability  to  separate  the  complete  effort  into  smaller,  concise  work  packages.  I lie  reasons  Un- 
doing this  are  several : 

I lie  job  is  much  easier  to  estimate  In  segment  than  as  a total. 

Hie  work  packages  essentially  become  the  task  breakouts  in  the  final  SOW 

[’he  prospective  contiactors  will  segment  the  work  in  their  responses,  giving  another 
dimension  for  evaluation. 

In  many  instances,  work  packages  can  be  used  to  indicate  a desired  approach  to  a 
problem. 

Che  action  of  generating  work  packages  w ill  indicate  to  the  technical  code  areas  m 
which  it  is  not  sure  of  the  specifics  of  its  requirements. 
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The  breakout  of  work  packages  cannot  be  formalized  into  a procedure,  since  each 
[ one  will  be  different  from  all  others  just  as  each  SOW  is  dilterent.  The  effort  can  only  be 

defined  as  a realistic  separation  of  a total  effort  into  its  reasonable  parts.  In  general,  the 
work  packages  should  conform  to  the  project  work  breakdown  structure  (W'BS)  (see  chapter 
I III.  Program  Planning  Organization). 

SOW  TASK  AREAS 

The  content  of  specifications  is  limited  to  technical  requirements  in  terms  of  design. 

I performance,  and  test.  I he  exactness  ot  design  details  may  be  specified  to  the  extent  neces- 

sary to  ensure  the  interchangeability  of  certain  components,  modules,  or  parts.  In  view  ot 
these  limitations,  it  becomes  necessary  to  establish  other  contract  tasks  in  the  SOW  to  sup- 
plement the  specification.  The  following  task  areas  are  typical  in  support  of  a specification: 

1.  Integrated  Logistic  Support  Program  Requirements 

2.  Configuration  Management  Program 

3.  Technical  Manual  and  Publications 

4.  Training  Requirements 

5.  Acquisition  Management  System  Requirements 
b.  Supply  Support  T asks 

7.  Engineering  Drawings  and  Associated  1 ists.  Tasks  and  Ordering  Data  (para  (>.2 
of  MlL-D-1000) 

8.  Contractor  Services  Requirements 

l>.  Quality  Program  Requirements 

10.  Level  of  Repair  Program  Requirements 

I I.  Test  Point.  Nodes.  Calibration,  and  Instrumentation  Information  Requirements 

12.  Reliability  Program  Requirements 

13.  Maintainability  Program  Requirements 

14.  Human  Factors  Program  Requirements 

15.  Safety  ‘Program  Requirements 

1(>.  PMS ^Planned  Maintenance  Subsystems)  Requirements 

17.  Electromagnetic  Compatibility  Program  Requirements 

18.  IT  MPEST  Control  (Program  Requirements 

In  addition,  there  are  project  tasks  which  go  beyond  the  scope  of  the  technical  el- 
lort  and  its  management  which  also  must  be  included  in  the  SOW  1 he  following  task  areas 
suggest  some  ot  these  “other'  tasks: 

I . The  development  of  specifications  and  data  for  future  project  phases 
2 Tasks  in  support  of  a desigu-to-cost  et t or t or  life-cycle  cost  procurement 

3.  Standardization  program 

4.  Value  engineering  program 
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5.  Special  research.  investigation,  or  study  i.isks  independent  of.  hut  supporting, 
the  equipment  specified 

(>.  Special  data  information  requirements  (which  will  he  ordered  in  the  C'DRLt 
not  derived  from  a specification  requirement  or  SOW  program  requirement 
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Statement  of  Work 
Brand  Name  or  I qual 

f lectric  Motor-Generator  Set.  (>0  11/  to  400  11/ 
\Y/  Company.  Model  I 244.  or  Iqual 


1.0  Introduction 

I I fills  statement  of  work  specifies  an  NOSO  requirement  for  fabrication  and  delivery 
of  a commercial  electric  motor-generator  set.  <i0  11/  to  400  II/.  1 lie  \YZ  Company  Model 

1 244  motor-generator,  or  an  equal,  will  meet  this  requirement. 

2.0  I echnical  Requirements 

2 I I he  contractor  shall  provide  all  necessary  materials  and  services  to  fabricate  an 
electric  motor-generator  to  meet  the  specified  characteristics  and  limits  in  the  attached 
\(  )S(  Specification  0004 "’SO. 
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APPENDIX  I)  EVALUATING  PROPOSALS 

The  purpose  of  the  evaluation  is  to  seleet  the  soureets)  whose  proposal  lias  the  high- 
est degree  of  realism  and  credibility  and  whose  performance  is  expected  to  best  meet  the 
government  objectives  at  an  affordable  cost.  This  is  to  be  done  in  a manner  which  assures 
an  impartial,  equitable,  and  comprehensive  evaluation  of  each  competitor's  proposals  and 
related  capabilities  and  which  minimizes  the  complexity  of  the  solicitation  while  maximiz- 
ing the  efficiency  of  the  evaluation  selection  decision.  I his  process  is  directed  In  Dot) 
Directive  4 105. b2  of  t>  January  D>7(>. 

The  evaluation  process  includes  an  evaluation  plan,  a solicitation,  the  actual  evalua- 
tion. negotiations'^)  clarify  details,  and  source  selection.  The  evaluation  plan  is  formulated 
prior  to  the  solicitation  and  serves  the  following  purposes. 

fo  ensure  that  all  efforts  are  directed  toward  a common  goal 

To  collect,  organize,  and  display  the  performance,  schedule,  and  cost  requirements 
by  emphasizing  pertinent  evaluation  criteria 

l o provide  a structure  for  organizing  the  evaluation  group  and  scheduling  its 
activities 

fo  provide  a structure  for  the  preparation  of  the  REE 

fo  establish  a format  for  discussion  at  preproposal  conferences,  it  held,  and  later 
offeror  or  contractor  discussions 

To  serve  as  a guide  for  the  contracting  authority  in  source  selection 
fo  pro\  uk'  procedures  and  methodology  lor  ev  aluation  purposes 

To  provide  guidelines  for  making  tradeoffs  among  and  within  the  various  factors  to 
the  performance  of  the  equipment  and  to  the  management  of  the  project  in  relation 
ship  to  the  development,  production,  operating  and  support  costs,  the  delivery 
schedule  and  quantity,  and  the  qualitative  requirements  of  the  procurement 

The  solicitation  should  request  proposals  in  two  parts,  the  first  containing  the  technical  and 
management  responses  and  the  second  addressing  costs.  Page  limitations  on  the  solicitation 
and  on  the  proposals  are  encouraged,  provided  that  completeness  is  not  sacrificed.  I he 
solicitation  should  state  technical  goals,  tolerances,  and  acceptable  values  within  which 
tradeoffs  can  be  made.  The  solicitation  also  must  request  the  ty  pes  ot  detailed  inlormalion 
which  will  allow  an  evaluation  of  the  technical  approach,  the  respondee's  understanding  ol 
the  complexity  of  effort  and  risks,  the  proposer’s  experience  and  capabilities  to  perform  the 
required  tasks,  and  the  realism  of  schedules  and  costs,  l he  solicitation  also  states  the  crite- 
ria. methods,  and  procedures  used  to  conduct  the  evaluation,  but  not  the  weighting  and 
scoring. 

I he  actual  evaluation  is  based  upon  the  scoring  of  each  proposal  against  the  prees- 
tablished evaluation  criteria  ranked  in  order  of  importance  and  weighted  accordingly  Spe- 
cific factors  and  rankings  will  vary  with  each  procurement,  but  the  cost  proposal  will  always 
be  evaluated  separately  from  the  other  part.  In  general,  the  scoring  should  follow  these 
guidelines: 

Raw  Score  Description 

‘>-1 0 (lH)-l  001  I xcellent  comprehensive  and  complete,  meets  or 

exceeds  all  proposal  requirements:  exemplifies  complete 


\\  i :s 


Raw  Score 


7-N  ( 7U-S0) 


5-<>  ( 5 ( )-<>()  > 


0 < 0-4l>  > 


Description 

understanding  of  the  requirements:  and  demonstrates  in 
detail  how  to  accomplish  the  task. 

Ciood  generally  meets  or  exceeds  proposal  require- 
ments: omissions  are  of  minor  consequence  or  small: 
would  be  likely  to  produce  an  acceptable  end  item. 

Adequate  omissions  are  of  significance,  but  are  cor- 
rectable; substantiation  of  points  is  weak  or  lacking: 
probability  of  successful  effort  is  marginal. 

Unacceptable  gross  omissions:  failure  to  understand 
problem  areas;  failure  to  respond  to  requirements:  little 
or  no  chance  of  success  in  completing  the  end  item. 


I ach  evaluator  will  formulate  questions  on  each  proposal,  and  the  consolidated  questions 
can  be  submitted  to  the  offeror  for  clarification.  In  each  case,  the  government  evaluates 
each  proposal  against  its  own  previous  estimates  to  assess  the  realism  of  cost,  schedule,  risk 
assessment,  and  technical  approach:  and  against  established  standards  for  management, 
accounting  practices,  and  the  like.  Proposals  which  arc  unrealistic  in  terms  of  technical  and 
schedule  commitments  or  unrealisticalU  low  in  cost  or  pr1  * can  be  rejected  on  the  grounds 
that  the  offeror  fails  to  comprehend  the  complexities  and  risks  of  the  contract  requirements 
or  else  has  made  an  improvident  proposal.  Only  proposals  which  are  evaluated  as  acceptable 
(by  preestablished  criteria  i will  be  passed  for  final  source  selection. 

file  final  source  selection  is  an  integrated  decision  based  on  a consideration  of  tech- 
nical approach,  capability,  management,  design-to-cost,  historical  performance,  price  cost, 
and  other  pertinent  factors.  The  selected  source(s)  should  be  the  one(s)  who  is  expected  to 
do  'lie  best  overall  job  for  the  government.  I he  selected  offeror's  proposal  (technical,  man- 
agement. and  cost)  must  satisfy  the  government's  minimum  requirement. 
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It  is  desirable  to  avoid  development  with  its  associated  risks,  tests,  and  time  re- 
quirements whenever  possible.  Nevertheless,  some  development  may  he  necessary  to 
support  the  various  phases  of  the  acquisition  cycle. 

DEVELOPMENT  IN  THE  CONCEPTUAL  PHASE 

- 

The  purpose  of  exploratory  development  is  to  investigate  or  evaluate  the  feasi- 
bility or  practicality  ol  a concept,  device,  circuit,  or  equipment  in  rough  experimental 
breadboard/brassboard  form  without  regard  to  the  eventual  overall  tit  or  final  form.  A 
concept  can  frequently  be  proved  in  theory  through  paper  analysis,  simulation  tech- 
niques, or  the  mere  existence  of  similar  commercial  products;  however,  these  techniques 
may  have  an  unacceptable  degree  of  uncertainty  attached  to  them  or  may  restrict  pro- 
gram options  unnecessarily.  Exploratory  development  as  a method  of  proving  feasibility 
is  relatively  inexpensive  compared  to  later  development  efforts  and  may  be  pursued  to 
excellent  advantage  if  certain  guidelines  are  followed  to  avoid  pitfalls.  In  combination 
with  some  degree  of  operations/threat/technology /force  objectives  analysis,  exploratory 
development  can  also  serve  to  validate  the  operational  requirements  statement. 

The  major  problem  to  be  solved  in  setting  up  an  exploratory  development  pro- 
gram is  to  maintain  options  in  the  follow-on  phases;  all  too  often,  a course  of  action 
picked  in  pursuing  exploratory  development  will  dictate  the  future  of  the  entire  acquisi- 
tion program.  The  primary  alternatives  in  conceptual  phase  development  are  the  following: 

In-house  development 

In-house  development  with  industry  support 

Industry  development  of  technology 

Industry  development  of  the  technical  approach 

The  in-house  alternatives  are  flexible;  when  industry  support  is  required,  small, 
well  defined  work  packages  should  be  formulated.  The  primary  difficulty  with  in-house 
approaches  is  in  creating  and  maintaining  a diversity  of  possible  technical  approaches  be- 
cause of  the  natural  biases  of  any  single  organization,  nevertheless,  a number  of  different 
approaches  can  be  generated  quite  efficiently  if  that  requirement  is  stated  in  the  develop- 
ment tasking.  When  industry  is  utilized  to  develop  the  technical  approach,  a number  of 
contractors  should  be  utilized  to  engender  competition  between  approaches.  The  eon- 
tracts  should  procure  all  rights  and  documentation  to  the  developed  approaches;  other- 
wise, the  government  may  later  find  itself  in  a sole-source  position  with  the  contractor 
dictating  the  prices.  Multisource  contracting  should  also  be  employed  when  industry  is 
used  to  develop  technology;  again,  the  procurement  of  rights  and  documentation  is  im- 
portant because  the  high  exploratory  development  risks  may  yield  only  a single  success- 
ful technology  which  the  government  may  wish  to  exploit  with  multiple  sources  in  the 
future.  > 

In  any  of  the  alternatives,  it  is  important  to  document  and  maintain  some  con- 
figuration control  on  the  developed  item  so  that  the  knowledge  gained  is  not  lost  to 
future  phases.  Another  important  consideration  is  the  decision  for  in-house  or  industry 
development  in  any  future  phase  of  the  acquisition  cycle.  If  industry  plays  a major 
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role  in  the  conceptual  phase,  it  is  difficult  to  pursue  any  in-house  development  later 
regardless  of  the  program  decision  factors  because  of  the  political  forces  which  are  ac- 
tivated by  industry  participation. 

When  it  is  necessary  to  sole-source  industry  participation  in  exploratory  develop- 
ment. the  participation  should  be  on  only  one  of  several  possible  alternative  approaches. 
Because  of  the  potent  political  forces  accompanying  industry  involvement,  the  industry 
tasks  should  be  divorced  from  the  program  in  name  and  funding  that  is.  totally. 

DEVELOPMENT  IN  THE  VALIDATION  PHASE 

Advanced  development  is  intended  to  demonstrate  the  technical  feasibility  of 
a design,  to  determine  its  ability  to  meet  existing  performance  requirements,  to  secure 
engineering  data  for  use  in  further  development,  and,  where  appropriate,  to  establish  the 
technical  requirements  for  contract  definition.  Dependent  on  the  complexity  of  the  equip- 
ment. technical  risks,  and  other  technological  factors  involved,  advanced  development  may 
necessarily  be  an  iterative  process  in  order  to  achieve  the  development  objectives.  The 
final  development  iteration  should  closely  approximate  the  required  form  factor  including 
the  appropriate  levels  ot  repair,  standardization,  reliability,  maintainability,  safety,  human 
engineering,  and  environmental  qualifications.  The  development  model  produced  may  be 
nothing  more  than  an  assembly  of  existing  equipments  or  it  may  consist  of  extensively 
developed  equipments. 

The  purposes  of  the  validation  phase  may  be  served  by  the  proof  of  feasibility  of 
a key  concept,  device,  circuit,  or  equipment  in  combination  with  technical  analysis  if  the 
other  portions  of  the  envisioned  system  are  well  defined,  the  system  integration  has  a low 
risk  associated  with  it.  and  sufficient  engineering  data  are  available  for  further  develop- 
ment and  any  contract  definition  requirements.  However,  the  confidence  achieved  through 
advanced  development  can  greatly  reduce  the  technical  risk  associated  with  follow-on  dev- 
elopment. Except  that  the  assembled  system  and  its  performance  to  existing  requirements 
are  at  issue  rather  than  the  feasibility  of  a piece  of  the  system,  the  considerations,  issues, 
and  alternatives  in  advanced  development  are  almost  identical  to  those  in  exploratory  dev- 
elopment. The  provision  of  documentation  and  preservation  of  options  in  the  succeeding 
acquisition  phases  are  important.  To  this  end.  compatibility  with  possible  procurement 
alternatives  and  the  major  issues  in  the  transition  to  production  must  be  maintained.  Since 
advanced  development  frequently  defines  values  for  parameters  in  the  technical  require- 
ments, it  is  important  also  to  define  acceptable  tolerances  in  order  not  to  exclude  pos- 
sible existing  equipments. 

When  industry  development  is  pursued,  competition  should  be  maintained  between 
at  least  as  many  sources  as  will  be  solicited  in  the  follow-on  phase. 


ENGINEERING  DEVELOPMENT 

Engineering  development  is  the  most  expensive  development  phase:  however,  the 
technical  risks  encountered  should  be  less  than  in  either  advanced  or  exploratory  develop- 
ment. Engineering  development  includes  the  product  design  and  production  engineering 
tasks  required  to  make  the  item  reproducible  while  maintaining  the  performance,  relia- 
bility. maintainability,  environmental,  human  engineering,  etc.  characteristics  which  were 
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determined  to  be  required  in  the  validation  phase.  The  technical  risks  are  greatly  increased, 
however,  when  an  advanced  development  phase  has  not  been  completed  or  when  the  tech- 
nical requirements  parameters  have  not  been  sufficiently  validated  to  have  established  ac- 
ceptable tolerances.  Tight  tolerances  are  more  difficult  to  engineer  into  production  proc- 
esses, and  the  lack  of  solid  engineering  data  increases  the  "fudge  factors”  an  engineer  uti- 
lizes to  ensure  the  product  will  perform  as  intended.  In  practice,  some  increased  risk  must 
be  assumed  in  order  to  move  ahead  in  the  acquisition  process.  ID  costs  are  higher  because 
of  (1)  the  scope  of  the  engineering  effort.  (2)  the  detailed  configuration  control  which  must 
be  maintained  on  production  processes  and  tooling  as  well  as  on  the  design  of  the  equip- 
ment. and  (3)  the  multitude  of  documentation  and  support  efforts  normally  associated 
with  this  phase  ILS  tasks,  technical  manuals,  provisioning  lists,  etc  (see  chapter  VI. 
Transition  to  Production). 

The  selection  of  an  engineering  development  alternative  will  depend  on  the  resolu- 
tion of  the  major  issues  in  the  transition  to  production  and  the  procurement  mode  alter- 
natives. The  first  issue  will  be  in-house  versus  industry  development:  the  factors  in  this 
decision  include  the  extent  of  industry  involvement  in  prior  phases,  the  appropriateness  ot 
in-house  involvement,  the  capabilities  of  available  in-house  resources  versus  industry 
sources,  the  government  needs  for  precise  control  of  the  technical  effort  in  conjunction 
with  other  program  factors,  and  the  percent  of  the  system  requiring  development.  If  in- 
dustry has  played  illy  a support  role  in  prior  phases,  in-house  involvement  is  appropriate 
(necessary),  and  capable  in-house  resources  are  available,  the  in-house  execution  should  be 
selected.  The  necessity  of  in-house  involvement  may  arise  from  the  factors  under  which 
in-house  development  is  appropriate  (see  chapter  VI).  from  government  need  for  control, 
or  from  the  primacy  of  the  system  integration  responsibilities  when  only  a small  portion 
of  the  system  requires  development.  Normally,  an  industry  alternative  will  be  selected: 
this  will  require  resolution  of  the  major  issues  prior  to  the  development  contract  solicita- 
tion. In  addition,  the  following  issues  must  be  considered:  (1  ) requirements  tor  compe- 
tition (form  a list  of  1.  2.  and  3).  (2)  program  cost  constraints,  and  (3)  prior  industry  in- 
volvement. 

If  industry  has  been  involved  in  prior  phases,  will  those  contractors  be  capable  id' 
meeting  the  remaining  program  requirements  for  production  and  support,  and  will  they 
be  adequate  in  number  to  support  any  needed  competition  dictated  by  the  procurement 
mode.’  A competitive  base  must  be  maintained  up  to  the  point  that  all  follow-on  cost- 
tor  procurement,  reprocurement,  and  contractor  support  sail  be  fixed  within  contractually 
binding  limits  If  new  contractors  are  to  be  brought  in.  provisions  must  be  made  to  edu- 
cate indoctrinate  the  contractors  to  the  program  requirements,  key  program  elements, 
acceptable  tradeoffs  which  may  not  have  been  specified,  etc.  If  a sufficient  number  of 
contractors  have  participated  in  advanced  development,  competitive  selection  may  he 
considered  for  engineering  development.  If  an  insufficient  number  of  contractors  are 
available  to  support  the  procurement  mode  needs,  provisions  should  be  made  to  execute 
the  preproduction  phase  in  house,  or  at  least  to  make  the  successful  validation  ol  the 
design  data  package  part  of  the  acceptance  procedure  for  the  I D contract. 

f urther  requirements  for  competition  may  arise  from  the  risks  associated  with 
the  technical  approaches  identified  in  the  prior  phases.  If  the  technical  risk  is  low  ie. 
the  major  system  equipments  have  been  previously  developed  and  used  together  on  a 
prior  program  and  require  only  adaptational  development  for  this  program  a single 
technical  approach  may  be  pursued.  If  a moderate  risk  exists  eg.  the  major  sy  stem 
equipments  have  been  previously  developed  but  never  before  integrated  in  this  way  a 
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primary  technical  approach  should  suffice,  but  alternative  approaches  should  be  identified 
which  can  provide  an  acceptable  product  with  minimal  transitional  costs  sho  Id  the  pre- 
ferred approach  bog  down.  If  a high  risk  exists  - eg,  one  or  more  major  system  equip- 
ments requires  development  parallel  technical  approaches  should  be  pursued. 

Program  cost  constraints  can  be  major  determining  factors,  especially  for  low- 
priority  programs.  Funds  may  not  be  available  to  support  competitive  development  or 
an  adequate  preproduction  (design  validation)  phase,  and  conditions  may  not  lend  them- 
selves to  functional  specification  procurement.  Assuming  in-house  development  cannot 
be  followed  even  though  it  may  be  the  best  alternative  under  these  conditions,  and  as- 
suming no  additional  funds  can  be  identified  for  design  validation,  the  procurement  must 
be  converted  to  a combined  design-to-cost  and  life-eycle-cost  procurement.  Design-to- 
cost  procurements  attempt  to  control  acquisition  costs  and  contractor-supplied  support 
costs;  life-cycle-cost  procurements  attempt  to  control  operations  and  support  costs.  Both 
types  of  procurement  are  discussed  below.  This  type  of  combined  pto,  urement  is 
risky;  the  contractor  must  be  contractually  obligated  to  absorb  loss  due  to  his  failure. 

If  such  a contract  is  not  feasible  (because  of  the  extent  of  development  uncertainty), 
and  if  contractor  default  cannot  be  allowed  for  any  reason,  the  government  is  better 
off  canceling  the  acquisition  altogether. 


DESIGN-TO-COST  (DTC)  PROCUREMENT 

Program  cost  constraints  in  the  procurement  phase  give  rise  to  design-to-cost 
procurement  modes.  In  DTC,  acquisition  costs  are  fixed  prior  to  engineering  development 
and  provisions  for  proving,  the  cost  targets  have  been  achieved  and  are  incorporated  into 
the  acceptance  procedures;  contractual  incentives  are  often  employed.  Very  wide  and 
notably  achievable  performance  tolerances  are  normally  required  because  cost  is  the  prime 
design  factor  and  performance  is  traded  off  to  achieve  that  cost  within  acceptable  limits 
The  advanced  development  phase  is  usually  required  to  identify  reasonable-cost  targets 
and  acceptable  performance  parameters. 

The  Joint  Design-to-Cost  Guide  (AMCP  700-o/NAVM AT  P5242/AFLCP-AFSCP 
800-19)  contains  guidelines  in  the  formulation  and  execution  of  a design-to-cost  procure- 
ment. 

See  Do  I)  Directive  5000.28. 


LIFE-CYCLE-COST  (LCC)  PROCUREMENT 

Life-cycle-cost  procurements  are  useful  when  the  equipment  service  life  is  long 
enough  that  support  costs  become  significant;  generally,  only  a few  years  is  required, 
as  these  costs  can  accumulate  rapidly.  A reasonably  accurate  LCC  model  tor  the  system 
is  required,  and  the  award  of  the  contract  is  made  to  the  lowest  bid  which  results  in 
the  least  overall  cost  to  the  government.  A number  of  procurement  modes  are  available 
along  this  theme.  The  Life  Cycle  Costing  Procurement  Guide  (Dol)  LCC- 1 ) is  use lul 
in  constructing  the  model,  and  the  Life  Cycle  Costing  Guide  for  System  Acquisitions 
(Dol)  LCC-3)  discusses  the  procedural  aspects,  including  some  suggested  procurement 
modes.  LCC  procurement  can  be  particularly  effective  when  long-term  warrants'  pro- 
visions are  integrated  into  the  procurement. 
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XVIII.  INSTALLATION  PLANNING 


Will.  INSTALLATION  PLANNING 

This  section  prov  ides  information  to  assist  the  project  manager  in 
planning  for  system  and  equipment  installations  on  Navy  platforms  tem- 
porary. for  tests  and  evaluations;  and  permanent,  for  service  use. 


TEMPORARY  INSTALLATIONS 

The  primary  purpose  of  temporary  installations  ( under  1 year)  is  to  subject  equip- 
ments to  the  operational  environment  fora  realistic  test  and  evaluation  (see  chapter  XIX). 
However,  short-term  operational  requirements  may  also  dictate  the  temporary  installation 
of  equipment  in  service  platforms,  and  occasionally  it  will  be  necessary  to  install  instru- 
ments to  measure  operational  environments.  Whatever  the  circumstance,  the  proper  man- 
agement of  a temporary  installation  is  important  to  the  success  of  the  project  and  to  the 
operation  of  targeted  platform. 

The  project  is  responsible  for  supplying  all  the  resources  necessary  to  accomplish 
and  to  disestablish  a temporary  installation.  The  funding  is  in  excess  of  any  special  operat- 
ing and  observer  costs  for  operational  tests  and  evaluation.  The  installation  personnel  are 
under  project  tasking.  It  is  incumbent  on  the  project  manager  to  plan  for  these  require- 
ments well  ahead  to  ensure  that  the  budget  is  adequate  to  support  the  installation  lasio 

The  most  difficult  task  in  the  temporary  installation  is  obtaining  Elect  services  and 
coordinating  them  with  the  availability  of  the  equipment  to  be  installed.  Elect  services  are 
requested  through  ( NO  in  accordance  with  OPNAVIX'ST  10.  (However,  minor  serv- 
ices are  sometimes  available  informally  through  special  laboratory  ty  pe  commander  relation- 
ships such  as  the  Navy  Science  Assistance  Program  (NSAP)  and  other  Elect  assistance  func- 
tions.) These  formal  procedures  are  designed  to  coordinate  the  burden  on  Elect  operations 
and  should  be  followed  for  all  formal  test  programs. 

The  other  tasks  are  as  follows: 

1.  Preliminary  planning  establishes  the  scope  of  the  installation:  the  number  ol 
equipments  involved;  the  interfaces  to  platform  systems  such  as  power,  air 
conditioning,  navigation  sy  stems,  and  sensors:  the  types  of  materials  needed: 
the  skills  needed  to  execute  the  installation:  the  ty  pes  of  support  required  for 
the  equipment  while  it  is  installed;  and  the  travel  and  transportation  require- 
ments for  the  installation  personnel  and  the  equipment. 

2.  Installation  survey  accomplished  well  ahead  of  the  planned  installation  date, 
the  survey  identifies  precise  equipment  installation  sites,  mounting  require- 
ments. relationships  with  platform  interfaces,  cable  routings  and  distances,  and 
man-hour  estimates  for  the  different  installation  skills 

.V  Detailed  planning  tasking  of  the  installation  team  (project  personnel,  con- 
tractor field  representatives,  shipyard,  etc ),  procurement  of  required  matenaE. 
documenting  the  installation  procedures,  establishing  checkout  procedures,  and 
determining  ripout  procedures. 

i Plans  for  temporary  submarine  installations  must  be  submitted  to  the  cogm 
/ant  NAVSI  A Ship  I ogistics  Managei  (SI  Mi  lor  approval  ) 

-I  Installation  and  checkout 
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5.  Ripout  il  was,  alter  all.  a temporary  installation. 

It  is  wise  to  use  the  installation  survey  to  establish  good  relations  with  platform  personnel 
and  to  keep  them  informed  of  your  intentions.  They  can  be  a valuable  source  ot  intorma- 
tion  regarding  unsuspected  problems  or  unusual  platform  characteristics  and  can  also  pro- 
vide limited  assistance  in  the  installation,  operation,  and  maintenance  of  the  equipment 
beyond  the  well  defined  scope  of  operational  testing. 

No  matter  which  type  of  platform  ship,  aircraft,  vehicle,  or  whatever  is  in- 
volved. the  thorough  advanced  planning  and  detailed  execution  ot  a temporary  installation 
are  essential.  The  failure  to  plan  adequately  will  result  in  badly  slipped  schedules  or  poor 
equipment  operation.  Ihe  installation  plan  should  include  steps  to  ensure  that  the  installed 
equipment  does  not  interfere  with  (or.  conversely,  is  not  susceptible  to  inierterenee  by ) 
equipments  already  on  the  platform:  many  problems  can  be  created  by  mounting  equip- 
ments too  close  or  overloading  interfacing  systems  and  the  like  which  a sensible  approach 
can  avoid.  Also,  the  installation  plan  should  show  contingency  plans  covering  changes  in 
schedule,  in  either  equipment  or  platform  availability.  As  with  any  other  project  task,  plan 
ahead,  anticipate  and  minimize  risks,  and  execute  the  plan  to  maintain  the  pattern  ot 
success. 


PERMANENT  INSTALLATIONS 

Navy  operational  requirements  tend  to  grow  ever  more  complex.  New  systems  and 
equipments  and  improvements  to  existing  equipments  are  constantly  demanded.  I hus. 
Navy  platforms  are  constantly  undergoing  change.  Without  change  control,  installation  ot 
new  equipment  and  alternation  of  old  would  eventually  become  impossible.  I able  W lll-l 
summarizes  the  types  of  alterations  and  changes  affecting  Navy  platforms. 


TABU  XVIII-I.  ALTERA  I IONS  AND  CHANGE’S. 


Ty  pc  ol 

Cognizant 

Initiating 

funding 

Change 

SYSCOM 

Document 

Source 

Reference 

SIIIPAl  I 

Sponsoring 

Military 

SYSCOM 

PM  1 

IMP 

OPNAVINST  4720. 2D. 

Improvement 

NAVSI-AINST  4720. v 

Technical 

fVP 

IMP 

Mil  -STD-480, 

Improvement 
ORC  SIIIPAl  1 

PM  1 or  1 CP  as 
appropriate 
and  l.tr  of 
Justification 

VAVSLAINST  4720.4 

OKDAI I 

NAYS!  A 

TCP 

project  ( until  nor- 
ma) budgeting  oc- 
curs) IMP 

(see  SIIIPAl  1st 

SPA  L I 

\ AV'SI  \ OX 

ICP 

OMN IMP 

SSl’l  P4720  10 

AIRAI  1 

NAVA  IK 

1 CP 

OMN PAMN 

(\  \Y  \IKI\SI  4720  1 It ) 

lot  modifications 


win  : 
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Tv  pc  of 

Cognizant 

Initiating 

Funding 

Change 

SYSCOM 

Docu  nicni 

Source 

Reference 

SIM  ( OM  At  1 

NAVLl.FA 

HP 

OMN 

NAVI  1 1 NINSl  4720.1. 
NAVI  1 1 NINSl  1 1000.  lb 

Shore  site 
alteration 
1 Bl  SI P| 

NAVP.I  I N 

HP 

OMN 

NAVI  1 1 NINSl  1 1000.  lb 

Electronic 
field  change 

NAVI  l I N 

K P 

(see  MU  -P-l  7655 1 

Mil  -F-I7655C 

SHIP  VLT1  RATIONS  (SHIP ALTS) 

All  proposed  alterations  classified  as  military  improvements  and  teelinieal  im- 
provements to  the  ship,  to  hull  equipments,  or  affecting  spaee  layout  or  eon  figuration  are 
handled  In  Sill  P.  V LI.  Alterations  to  ordnance  equipment  lORDAI  Is)  are  handled  as 

SI  1 1 PAL  Is. 

Ship  alterations  are  controlled  through  the  I'leet  Modernization  Program  < IMP). 

I he  Naval  Sea  Systems  Command  (NAVSI  \)  is  the  designated  executive  agent  within 
\ ACM  A I for  the  IMP.  1 or  alterations  ol  minor  impact  (NO  delegates  responsibility 
for  configuration  control  to  the  cognizant  systems  command. 

Plicse  programs  are  structured  so  that  they  max  he  properly  planned.  d<  ument- 
ed.  scheduled,  funded,  and  executed.  Review  of  engineering  effort  and  cost-pci lormance 
tradeoff  are  built  in.  The  project  manager  is  well  advised  to  allow  adequate  time  lor  all 
these  essential  steps  in  what  by  its  nature  must  be  a lengthy  and  serial  procedure 


I I I I I MODERNIZATION  PROCR  \M 

Ship  alteration  programs  requiring  depot  level  capability  or  budget  action  lot  the 
procurement  of  special  material  are  scheduled  through  the  Meet  Moderni/alion  Program 
i IMP).  Ihe  IMP  is  promulgated  by  (NO  and  managed  by  N \\ SI  \ (see  fig  Will  I)  li 
is  implemented  via  the  semiannual  I I I MOD  conference  Inputs  to  the  conlercnce  are 
consolidated  in  the  Military  Improvement  Plan  (MIPl.  the  leeltnic.il  Improvement  Plan 
( I IP),  and  the  Amalgamated  MIP  IIP  (AM  I I one  \M  I lor  each  ship  class 

I he  Mil’s  and  I IIN  are  working  papers  used  by  the  S')  S(  ( INK  and  iv pe  command 
ers  m preparing  for  I I I M( )l)  conferences  and  by  the  Ship  Acquisition  and  Improvement 
Panel  (S AIP)  as  inputs  to  the  \M  Is. 

| lie  \M  Is  are  developed  by  the  SNIP  Working  Croup,  with  the  technical  sup- 
port of  ( I INAVM  \ I I hex  are  based  on  consideration  ol  ship  and  personnel  s.ifei y . new 
sv  stems  and  programs,  fleet  capability . and  ( N( ) policy  I hey  consolidate  and  prioritize 
MIP  and  I IP  items  and  make  it  possible  to  establish  realistic  plans  toi  then  accomplish 
me nt . I hey  are  used  to  determine  alterations  to  be  added  when  additional  funds  become 
available  and  alterations  to  be  eliminated  when  programs  are  reduced.  I hey  constitute  the 
basic  I \IP  documentation. 
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Figure  Will- 1,  hlcel  modernization  program  (F MP). 

SUBMITTING  PROPOSALS  FOR  ALTERATIONS 

Tile  type  ot'  initiating  document  which  must  he  submitted  for  ;i  proposed  altera- 
tion depends  upon  the  nature  of  the  alteration  (see  table  XVI1I-1  ).  The  proposed  Mili- 
tary Improvement  (PMI)  is  the  initiating  document  for  a SI  I IP  A l I which  effects  a mili- 
tary improvement  (one  of  nature  or  magnitude  such  as  to  increase  its  operational 
capabilities!,  flic  Engineering  Change  Proposal  ( TCP)  is  the  initiating  document  for  a 
SIIIPAL  I which  effects  a technical  improvement  (one  which  concerns  the  safety  of  per- 
sonnel or  the  reliability,  maintainability,  and  efficiency  of  installed  equipment ) 

Ordnance  alternations  (ORDALTs)  and  Special  Project  Alterations  ( SPA  1 Is) 

( nuclear  propulsion  plant  and  tender  nuclear  support  facility  alterations!  are  also  initiated 
by  ECP. 

It  is  important  that  the  project  manager  fully  identify  critical  installation  parame- 
ters and  procedures  upon  requesting  an  alteration.  I he  estimated  increase  in  weight  and 
vertical  moment  should  be  stated,  together  with  a recommendation  for  the  removal  ot 
sufficient  weight  to  compensate  for  it.  If  an  alteration  involves  a reduction  in  space  or 
liv  ing  facilities  normally  available,  the  amount  of  the  reduction  and  the  reason  for  accept 
mg  it  should  be  stated. 

After  a PMI  is  approved,  and  upon  request  of  the  appropriate  SYSCOM.  the  PMI 
originator  develops  information  to  serve  as  guidance  for  the  SYSCOM  in  matters  concern- 
ing plans,  support  requirements,  equipment  material,  test  equipment,  test  and  checkout, 
weight  and  moment,  training,  manpower,  and  personnel. 
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In  the  interests  of  standardization,  approved  alterations  are  normally  authorized 
for  all  ships  to  which  they  are  applicable. 


APPROVAL  OF  ALTERATIONS 


MILITARY  IMPROVEMENTS 

The  decision  to  incorporate  a Proposed  Military  Improvement  (PMl ) rests  with  ( NO 
(SA1P).  The  SA1P  Working  Group  reviews  and  determines  acceptance  at  the  I LTMOD 
conference.  Due  regard  is  given  to  ship  situation  resulting  from  alterations  already  included 
in  the  AMT  for  the  particular  ship  class. 

Approved  PMIs  are  entered  officially  in  the  MIP  and  are  considered  for  appropriate 
priority.  If  doubt  exists  as  to  whether  an  improvement  is  military  or  technical,  the  matter  is 
referred  to  ( NO  (SAIP)  for  resolution.  PMIs  not  approved  are  returned  to  the  originator. 


I I ( I INK  AL  IMPROVEMENTS 

Requests  for  approval  of  proposed  technical  improvements  are  forwarded  to  the 
cognizant  SYSCOMs  as  I CPs.  The  SYSCOM  reviews  the  alteration  under  the  command  of 
the  SYSCOM  Change  Control  Board  (CCB).  The  alteration  is  reviewed  for  concurrence  in 
classification.  The  CCB  approves  or  disapproves,  assigns  priorities,  and  determines  whether 
the  improvement  will  be  handled  as  a SHIPALT.  If  it  is  to  be  handled  as  a SI  1 IP  AL  I . the 
approved  1 CP  is  forwarded  to  NAVSEA.  where  it  is  othcially  entered  in  the  technical 
Improvement  Plan  (TIP)  upon  acceptance  and  considered  for  priority  at  the  next  I I I MOD 
conference.  The  I IP  is  approved  by  ( I1NAVMAT. 

Ordnance  All  -rations  (ORDA1  Is)  are  handled  in  much  the  same  manner,  by  the 
cognizant  ordnance  section  of  NAVSEASYSCOM. 

If  the  technical  improvement  is  an  Alteration-l'quivalent-to-Repair  (Al  R)  (substitu- 
tion of  different  but  standard  materials  or  parts  of  later  design : strengthening  tor  greater 
reliability ; or  other  minor  modification  or  replacement ),  it  may  be  approved  and  authorized 
for  accomplishment  by  the  fleet  commander  to  the  extent  that  such  authority  lias  been 
delegated  In  the  SYSCOM.  A SHIPALT  is  not  necessary  . 

When  a technical  improvement  is  disapproved,  the  originator  is  notified. 


EXECUTION  OF  Till  I MP 

I he  I I I MOD  conference  takes  place  each  year  in  May  and  November.  I he  I MP  is 
published  in  July  and  January  . Il  contains  firm,  proposed,  revised,  and  tentative  listings.  Ii 
constitutes  the  ( NO  approved  program  for  modification  and  improvement  ol  the  ships  ol 
the  licet. 

Fleet  and  ty  pe  commanders.  OPNAV.  and  N AYS1  ASYSCON1  and  other  commands 
participate  in  the  formulation  and  review  of  this  program.  NAYSI  \ collaborates  closely 
with  all  concerned  to  achieve  efficient  execution  ol  changes  in  requirements,  available  fund- 
ing. material  deliver,  schedules,  overhaul  schedules,  and  the  otliei  (actors  which  bear  upon 
stability  N \\  SI  \ issues  a Material  Supplement  for  each  semiannual  I MP  covering  the 
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I MP  I execution  and  Budget  Year  and  autliori/es  advance  planning  In  Planning  and  1 Engi- 
neering for  Repair  - and  Alterations  (PLRA)  and  overhaul  yards  .is  required. 

At  the  time  of  approval  and  promulgation  of  the  IMP  by  OPN  \V.  distribution  of 
the  I MP  is  made  under  the  cover  of  ( NO  letter  of  promulgation.  I his  letter  ad\  ises  the 
fleets  regarding  actions  to  be  taken  if  program  adjustments  become  necessary  to  meet  chang- 
ing operation  conditions. 

Requests  for  changes  to  the  approved  I MP  from  f orces  Afloat.  Program  Managers, 
or  other  sources  must  be  directed  to  ( NO.  with  compensation  and  justification,  via  the 
Ty  pe  and  f leet  Commander  where  applicable,  with  information  copies  to  NAVSI. A. 

QRC  RDC  SHIP  ALTS 

Requirements  which  are  identified  out  of  phase  with  the  normal  programming  and 
budgeting  cycle  and  which  are  considered  by  the  SAIP  Working  Group  to  be  of  higher 
priority  than  existing  requirements  are  accommodated  on  a case  basis.  Quick  Reaction 
Capability  SlIlPALTs  are  initiated  by  letter  of  justification.  Sponsors  of  these  Quick  Reac- 
[ (ion  Capability  and  Rapid  Development  Capability  requirements  must  assure  that  normal 

programming  and  budgeting  are  commenced  immediately. 

SHIP  LOGISTICS  DIV  ISION  APPROVAL 

All  new  shipboard  installations  and  configuration  changes  that  affect  the  interface 
between  a ship  and  its  systems  and  equipments  are  reviewed  by  the  cogni/ant  NAVSI  \ 

Ship  Logistics  Division  (SI  D).  This  includes  all  long-term  (over  I year!  temporary  installa- 
tions such  as  mission-oriented  installations.  OP  I ecli  Installations,  and  R&D  installations. 

I he  installation  plan  should  be  submitted  through  the  cogni/ant  ty  pe  commander 
well  in  advance  of  the  required  approval  date  in  order  to  allow  adequate  time  for  N AVSI  \ 
review.  It  should  include  the  following  information:  project  title,  security  classification, 
sponsoring  activity  . applicable  ship  and  type  commander,  installation  duration,  installation 
site,  installation  activity  , installation  technical  support  requirements,  installation  impact 
data,  and  installation  drawings  (see  appendix  Al. 

NAVSI  A reviews  the  installation  plan  tor  technical  adequacy  , technical  accuracy . 
and  impact  on  ship  safety  , personnel  safety  , military  capability  , sy  stem  interlace,  equipment 
arrangement,  stability . and  ship  sy  stems,  and  sends  a message  or  speed  letter  to  cognizant 
activities  on  approv al. 


\IH<  R \FT  \LTLR  \TIONS  I AIR  VLTSl 

Aircraft  alterations  and  changes  are  managed  by  the  Naval  An  Systems  Command 
( N AY  MR).  Proposed  alterations  are  submitted  via  I CP  to  the  cognizant  .ml i ante  ly  pe  desk 
m NAV  MR.  I he  NAVAIR  CCB  acts  on  each  1 CP  and  the  originator  is  notified.  Normally . 
approved  alterations  are  accumulated  and  incorporated  en  masse  into  the  airt/ame  as  a 
change  of  model  ( eg.  I -IB  becomes  I— M ) Mass  changes  are  coordinated  with-ne^v  aiicialt 
procurements  and  aircraft  overhaul  schedules.  Changes  which  can  be  accomplished  by 
direct  black  box  substitution  are  programmed  to  coincide  w ith  overhauls  Migrations  which 
are  safely  critical  are  scheduled  on  a priority  basis. 


W III  n 


r 


SHORE-  Sill  ALTERATIONS 


Shore  site  alterations  include  Base  Electronic  System  E ngineering  Plan  (BESEP)  and 
Special  Communications  System  Alterations  ( SPECOM AL  ES).  Both  are  administered  In 
the  Naval  Electronic  Systems  Command  (NAVI  LEX).  Proposed  changes  are  submitted  via 
ECP  and  acted  upon  by  the  NAVEI  EX  CCB.  BESEPs  are  specifically  managed  by  the 
NAVI  I EX  assigned  Field  Lechnical  Authority  (ETA).  NAVE. LEX  P.ME-1  17  manages 
SPECOMALTs.  (All  non-SIM  COM  alteration  proposals  should  be  submitted  to  NAVELEX 
headquarters:  SPECOM  alteration  proposals  should  be  directed  toPME-l  17.) 


FIELD  CH  ANGES 

Field  changes  are  developed  for  the  purpose  of  improving  equipment  after  delivery 
to  the  government  with  respect  to  performance,  operational  characteristics,  or  maintenance 
features,  file  change  may  be  a simple  wiring  or  mechanical  modification  to  an  existing 
equipment  and  consist  entirely  of  instructions  for  making  the  change,  or  it  may  be  more 
extensive,  requiring  circuit  changes  and  the  removal  and/or  substitution  of  parts.  Material 
supplied  or  required  is  listed  in  the  field  change  bulletin,  which  also  provides  step-by-step 
instructions  for  accomplishing  the  change. 

Field  changes  bear  designations  of  type,  class,  and  priority  which  are  defined  in 
MIE-E'-I  7655C.  Type  indicates  extent  to  which  parts  are  supplied:  class  indicates  funding 
and  installation  responsibility;  and  priority  indicates  urgency  of  accomplishment. 

Changes  are  initiated  by  the  submission  of  an  Engineering  Change  Proposal  ( ECP) 
(see  Mil  -S  I D-480  or  -481  ).  The  originator  may  be  any  agency  involved  in  the  design,  pro- 
duction. maintenance,  or  operation  of  a system  or  subsystem.  If  the  proposed  change  re- 
quires a related  change  to  equipment  or  material  under  the  cognizance  of  another  command 
a Request  for  Conjunctive  Alteration  is  submitted  with  the  ECP.  The  originator  must  en- 
sure that  appropriate  drawings,  sketches,  and  reference  data  are  submitted  to  the  Cogm/ant 
lechnical  Division,  the  Coordinating  Activity,  the  Change  Control  Board  (CCB).  and  other 
reviewing  activities.  The  originator  must  also  assure  that  copies  of  the  ECP  are  distributed 
simultaneously  to  all  reviewing  authorities. 

Ehe  SYSCOM  CCB  reviews  and  evaluates  proposed  changes  and  recommends  appro\ 
al  or  disapproval  to  the  cognizant  SYSCOM  Project  Office.  The  Project  Office  makes  the 
final  determination. 

I f an  I CP  is  disapproved,  all  affected  activities  are  notified. 


ECP  APPROVAL 

Ehe  systems  command  project  offices  and  CCBs  base  their  approval  upon  the  follow 
mg  factors: 

1.  Need  for  the  change,  adequacy  of  justification 

2.  Cost-effectiveness 
x.  I muling  availability 

4 Effects  on  training  installations 

5.  Impact  on  logistu  support 
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6.  Proposed  installation  schedule 

7.  Adequacy  of  the  design  and  procurement  documentation 

The  approval  or  disapproval  will  he  documented  by  the  ( C B and  forwarded  to  the 
originator. 


OTHLR  NOTLSON  INSTALLATIONS 

Installation  plans  contribute  to  the  equipment  technical  manuals.  Refer  to  MIL-M- 
15071  for  specific  instructions. 

In  formulating  installation  plans,  remember  safety  and  maintenance.  11k  install*)' 
tion  should  not  create  hazards  for  the  operators  and  maintainers.  and  it  should  provide  lor 
ample  maintenance  access. 


\ 
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APPENDIX  A:  INSTALLATION  PLAN 


The  objective  of  the  installation  plan  is  to  provide  all  the  necessary  information  to 
perform  a cost  and  feasibility  (C'&L)  study.  This  ensures  that  all  elements  impacting  the 
ship  and  its  various  systems  are  considered  and  identified.  This  information  is  made  availa- 
ble to  those  concerned  for  decision  making  purposes  during  the  process  leading  to  an  accom- 
plished ship  improvement.  It  must  be  recognized  that  the  validity  of  a C&F  study  depends 
upon  the  accuracy  of  the  technical  input;  therefore,  the  originator  is  responsible  for  provid- 
ing technical  data  of  the  highest  possible  validity. 

Valid  information  considering  each  of  the  following  items  is  desired  with  each  pro- 
posed improvement. 

I.  PLANS 

A.  Block  Diagrams.  Show  the  function  of  each  basic  unit  with  relation  to  other 
units  comprising  the  system.  Indicate  normal  inputs  and  outputs  of  each  unit 
in  the  system.  Include  block  diagrams  of  all  units  to  be  furnished  plus  all  other 
units  required  to  complete  the  installation,  including  racks,  junction  boxes, 
motor  generators,  and  antennas. 

B.  Arrangement.  Include  arrangement  for  each  electronic  space  and  for  all  other 
spaces  in  w hich  an  appreciable  amount  of  electronic  equipment  is  installed. 
Prepare  a plan  view,  an  elevation  view,  and  other  views  as  necessary  to  show  the 
arrangement  clearly.  Structural  members,  ventilation  ducts,  piping,  and  equip- 
ment other  than  electronic  equipment  shall  be  indicated.  Also  show  large  trans- 
mission lines,  (live  details  of  equipment  characteristics  that  influence  the  rela- 
tive locations  of  units  to  optimize  operation,  maintenance,  reliability  . and 
safety. 

('.  Stowage,  (live  ty  pical  stowage  requirements,  phy  sical  dimensions,  and  environ- 
mental requirements  for  major  spare  parts.  Include  stowage  area  for  test  equip- 
ment. special  tools,  technical  manuals,  etc. 

D.  Ripout 

1.  Removal  instructions.  Include  all  equipment,  cables,  piping,  and  founda- 
tions to  be  removed.  I lie  following  should  be  provided: 

a.  Location  of  item 

b.  Name  of  item 
c Disposition 

d Cable  run  and  length 

e Weight  ol  item  (note  change  of  weight  and  moment  I 

2.  ( ompartment  and  access  modification.  Include  all  structural  members, 
bulkheads,  etc.  to  be  removed.  All  drawings  should  resemble  arrangement 
drawings 

I Interlaces  Ml  interconnection  requirements  with  other  sets,  power  sources. 

eU  . shall  be  shown  and  identified.  Include  external  cables,  rf  transmission  lines, 
piping  lor  water  cooling  and  healing,  piping  for  air  and  other  services,  and  any 
types  ol  interconnections  among  units  and  between  units  and  power  sources. 
Identity  necessary  concurrent  or  prior  completion  items  such  as  Sllll’Al  Is. 
f ield  < liangcs.  and  Mods. 
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II. 


SUPPORT  REQUIREMENTS 


A.  Power.  Indicate  type  of  power  to  be  used.  For  electrical  power  give  the  require- 
ments for  the  complete  equipment  indicating  for  each  power  input:  power  type 

1.  II.  or  III  (as  defined  by  MI  E-STD-7U 1 ).  voltage,  phase,  frequency,  current, 
power  factor,  and  power.  Power  for  starting,  operating,  standby,  and  secured 
conditions,  as  applicable,  shall  be  specified.  Allowable  variations  in  voltage  and 
frequency  shall  also  be  specified. 

B.  Cooling.  Indicate  type  of  cooling  or  heating,  whether  water  or  air.  Give  the 
amount  of  heat  to  be  dissipated.  A diagram  should  depict  the  routing,  direction 
of  How.  components  with  physical  location,  arrangement,  and  other  characteris- 
tics or  constraints.  All  units  to  be  cooled  or  heated  shall  be  shown. 

C.  Air  conditioning.  Give  the  ambient  temperature  range  and  limits  of  relative 
humidity  for  which  the  equipment  is  designed.  Indicate  the  location,  si/e.  and 
arrangement  of  ventilation  openings  and  external  ventilation  ducts  supplied  or 
required,  indicating  capacity  and  direction  llow.  Interface  to  ship  services  shall 
be  clearly  specified  and  shall  include  duct  size,  material,  allowable  particle  si/e, 
allowable  moisture  environment,  allowable  velocity,  filter  characteristics,  and 
pressure  required. 

1).  Special  Material.  Make  a list  of  special  material  such  as  cables,  special  tools. 

waveguides,  mounting  equipment  and  hardware,  connectors,  bulk  material,  parts, 
loose  hardware,  test  equipment,  and  other  support  items  needed  to  complete  the 
installation  which  are  not  part  of  the  installation  package. 

E.  Shock.  Indicate  any  special  part  arrangements,  mounting  techniques,  or  joining 
methods  to  harden  equipment  against  shock  and  vibration.  Give  clearances  re- 
quired for  shock  and  vibration  excursions  on  shock-mounted  equipment. 

F.  Safety.  Establish  safety  requirements  to  promote  maximum  safety  for  personnel 
and  equipment  during  installation. 

1.  Identify  potential  hazards  and  methods  planned  to  eliminate  or  control 
them. 

2.  Outline  undefined  areas  requiring  guidance  or  decisions. 

3.  Describe  technical  risks  or  problems  in  design. 

G.  Stowage.  Prepare  a list  of  all  installation  material  such  as  spares,  technical 
manuals,  and  tools  to  be  stored  in  stowage  areas. 

II.  Planned  Maintenance  System  (PMS).  Express  the  planned  maintenance  require- 
ments of  the  system,  subsystem,  or  component  (see  Ol’N  \V  43P2  and 
OPNAVINST  4740.4).  The  planned  maintenance  requirements  should  include 
the  following:  cleaning,  inspection,  lubrication,  replacement,  functional  test, 
adjustment  and  alignment,  calibration,  and  systems  check  (see  Mil  -P-2S’’5'h 
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\ Description.  Give  the  weights  and  space  requirements  ol  the  new  and  replaced 
lit  an\  ) installation.  Mso  mention  an\  consti.imts  that  max  exist 
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B.  Special  Instructions.  Mention  all  special  instructions  such  as  handling  (location 
of  handling  attachments,  crating),  mounting  (shock  mounting,  special  clearances, 
mounting  surface  and  hardware),  cable  entrance  sway  bracing,  etc. 

( . Standard  Stock  Items  versus  New  (contract)  Items.  Consider  cost  tradeoffs  in 
logistic  requirements  for  supporting  any  new  mateiial  or  equipment  involved 
and  retrofitting  the  new  material  or  equipment  to  existing  systems  already- 
deployed. 

I).  Indication  of  Spare  Support  for  New  equipment.  I ist  the  sources  of  all  spare 
support  materials  and  equipment  whether  it  be  the  government  f ederal  Stock 
System  or  a contractor. 

I . Availability.  Make  a brief  statement  of  the  availability  status  of  the  equipment 
or  material.  Include  when  available  for  installation,  confidence  of  this  date, 
status  of  tech  op  evals  or  service  approval,  procurement  lead  times,  and  any 
constraints  that  could  modify  this  status. 

F.  Technical  Manuals.  Technical  Manuals  shall  be  available  upon  installation  date 
and  provide  information  such  as  instructions  for  installation,  operation,  and 
maintenance. 

IV.  11  SI  FQUIPMFNT 

A.  General  Purpose.  Prepare  a list  of  all  general-purpose  test  equipment  to  be  used. 
Include  the  official  name  or  nomenclature,  identifying  number,  and  a brief 
description  of  the  use  of  the  item. 

B.  Special.  Prepare  a list  of  all  special  tools  and  special  test  equipment  to  be  used. 
Special  tools  and  test  equipment  are  defined  as  those  not  listed  in  the  f ederal 
Supply  Catalog.  An  illustration  and  description  of  special  items  required  shall 
be  provided  for  identification. 

V.  II  ST  AND  CHKCKOU  I 


Prepare  installation  and  checkout  lest  plans  and  procedures  to  ensure  that  all  equip- 
ment is  physically  and  functionally  checked  out  and  to  demonstrate  the  adequacy  of  each 
facility  installation.  Information  will  be  obtained  from  such  sources  as  specifications, 
related  test  documentation  used  in  the  development  test  program,  and  relevant  technical 
documentation  from  any  agency  concerned  with  the  installation  testing. 

I he  test  plan  section  shall  describe  the  overall  installation  lest  program  and  should 

include: 

1.  Description  of  the  equipment  and  facility  configuration  at  each  site 

2.  Scope  of  testing 

,v  Objectives  of  each  test 

4.  Roles  and  responsibilities  ol  all  major  participants  including  the  government 

5.  Support  requirements  including  test  instrumentation  and  facilities 
().  Milestones  and  scheduling 

7.  Security  guidelines 
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Detailed  test  procedures  shall  be  provided  which  shall  include  the  following  classes  ot 

tests: 

a Preshakedown  Tests.  This  portion  shall  contain  instructions  tor  making 
physical  inspections,  turn-on  -off  procedures,  alignment,  adjustment,  and 
measurements  required  to  be  accomplished  prior  to  the  start  ot  shakedown 
testing  of  the  facility. 

)-,  Shakedown  Tests.  This  portion  shall  include  a tabulation  ot  the  equipment 
and  components  to  be  subjected  to  shakedown  tests,  the  total  population 
of  electronic  parts  in  each,  and  the  required  duration  of  the  test  for  each. 

c.  Operational  Tests.  This  portion  shall  include  instructions  tor  performing 
the  tests  required  to  demonstrate  that  all  equipment  programs  or  facilities 
covered  in  the  test  plan  and  work  specification  are  properly  installed  and 
are  capable  of  performing  their  operational  mission  up  to  the  prescribed 
interfaces  with  other  portions  ot  the  subsystem  and  system. 

VI.  WEIGHT  AND  MOMENT 

Include  the  following  when  determining  weight  and  moment  characteristics:  total 
weight,  weight  reference  to  baseline,  weight  reference  to  midpoint,  net  moment,  and  weight 
reference  to  centerline  (see  SUPSH1P1NST  9290  1C). 

VII.  TRAINING 

The  immediate  and  long-range  training  requirements  tor  both  civilian  and  militan 
personnel  must  be  considered.  State  the  following  requirements: 

1 . Training  courses  required  including  course  length. 

2.  Training  materials  and  facilities  required. 

3.  A summary  of  technical  data  required. 

4.  Instruction  advisory  services  required. 

VIII.  MANPOWER 

When  the  accomplishment  of  an  alteration  affects  any  aspect  ol  manning,  the  Ship 
Manning  Document  (SMD)  must  be  updated  to  reflect  the  resulting  changes.  Items  such  as 
the  addition,  removal,  or  modification  of  equipment  and  systems  may  necessitate  increases 
or  decreases  in  existing  manning  levels,  and  must  be  considered  in  view  ot  all  manning  tactois. 

IX.  PERSONNEL 

Number  and  type  to  install,  check  out.  operate,  and  maintain. 
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XIX.  TEST  AND  EVALUATION 

BACKGROUND  . . . page  XIX- 1 
PURPOSES  . . . XIX-1 
INSTRUCTIONS  . . . XIX-2 

GENERAL  PLANNING  FOR  EACH  PHASE  . . . XIX-2 
ESTIMATING  COSTS  OF  T&E  ...  XIX-1  1 
T&E  PLANNING  . . . XIX-1 1 

Test  and  Evaluation  of  Major  Systems  . . . XIX- 12 
T&E  Cycle  . . . XIX- 13 

T&E  Management  of  Other  Than  Major  Systems  . . . XIX- 15 
T&E  Management  of  Ship  Construction  . . . XIX-1 5 
Nuclear  Weapons  T&E  . . . XIX-1 5 


XIX.  TEST  AND  EVALUATION 


XIX.  TEST  AND  EVALUATION 
BACKGROUND 

DoD  Directive  5000.1  of  13  July  1971  establishes  a “fly  before  buy”  philosophy 
for  all  major  acquisition  programs  (over  $50  million  RDT&E  and/or  over  $200  million  in 
production);  this  policy  was  extended  by  SEC'NAV  Instruction  5000.1  of  13  March  1972 
to  all  Navy  acquisition  programs.  I his  has  resulted  in  revisions  to  the  policies  for  granting 
service  approval  making  Approval  lor  Service  Use  (ASU)  contingent  upon  a satisfactory 
completion  of  a test  and  evaluation  program  executed  by  COMOPTEV1 OR  who  has  been 
assigned  responsibilities  as  the  Navy’s  independent  test  agency  (see  chapter  VII.  Approval 
for  Service  Use).  Service  approval  is  a primary  milestone  in  satisfying  an  operational 
requirement;  however,  each  of  the  steps  in  the  acquisition  cycle  has  test  and  evaluation 
(T&E)  needs  to  be  met  even  if  no  mandatory  policy  exists. 


PURPOSES 

There  are  seven  basic  purposes  for  executing  a T&E  program  defined  as  follows: 

1.  Investigation.  Examines  natural  oi  special  phenomena  in  an  operational  en- 
vironment or  gathers  data  to  determine  preferred  technical  alternatives, 

2.  Diagnosis.  Determines  the  origins  and  nature  of  undesirable  behavior  observed 
or  anticipated  in  a test  item. 

3.  Evaluation.  Appraises  the  parameters  and  attributes  of  a test  item. 

4.  Verification,  c onfirms  the  achievement  of  an  established  level  of  operation, 
the  suitability  of  interfacing  facets  (space,  fixtures,  power,  etc),  or  the  ade- 
quacy of  descriptive  item  documentation. 

5.  Qualification.  Proves  the  capability  of  the  test  item  of  meeting  established 
requirements. 

6.  Acceptance.  Determines  conformance  to  specification  requirements  prior  to 
acceptance. 

7.  Appraisal.  Determines  optimum  procedures  and  tactics,  confirms  corrected 
discrepancies,  and  verifies  suspected  discrepancies. 

Each  phase  of  the  acquisition  cycle  may  draw  on  one  or  more  test  purposes,  investiga- 
tions are  used  to  support  the  Requirements  Definition.  Concept  formulation.  Validation, 
and  Screening  phases.  In  the  early  phases  the  investigation  serves  to  better  define  the 
operational  and  technical  requirements;  in  later  phases  it  serves  the  purpose  of  selecting 
technical  alternatives.  Appraisals  may  be  performed  on  production  equipments  to  verify 
the  existence  or  correction  of  discrepancies  noted  in  operational  evaluation  (OP!  V \l  ) or 
discovered  in  fleet  operations.  Appraisals  arc  also  conducted  to  determine  optimum  pio- 
ecdures  and  tactics  for  utilizing  new  capabilities  or  adapting  existing  capabilities  to  new 
threats;  in  this  latter  vein,  they  may  also  be  conducted  to  better  define  and  determine 
requirements  for  future  system  acquisitions.  Since  appraisals  arc  not  part  ol  any  particu- 
lar acquisition  program  because  they  are  conducted  directly  in  support  of  ( \()  tin  \ am 
not  discussed  further  here.  I he  other  test  purposes  serve  tile  program  phases  shown  in 
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table  X1X-1.  Notice  that  diagnostic  tests  are  only  performed  in  conjunction  with  other 
types  of  tests;  this  type  of  test  serves  to  provide  the  analysis  data  to  support  corrective 
engineering  efforts.  When  structuring  a test  program,  certain  elements  must  f.  present  to 
accomplish  the  purposes  of  the  testing;  these  elements  are  discussed  separately  lor  \nh 
purpose  later  in  this  chapter. 


INSTRUCTIONS 

In  planning  for  test  and  evaluation,  there  are  two  vital  instructions 
Test  and  Evaluation  OPNAV1NST  3960.10. 

Mission  and  functions  of  Operational  Test  and  1 valuation  force  lOl’ll  \ I (>Kt 
OPNAV1NST  5440.47  (series). 

OPNAV1NSI  3960.10  detines  the  types  of  priorities  and  services  available  sets  forth  pro- 
cedures for  the  prosecution  of  test  and  evaluation  programs,  establishes  Navy  test  and 
evaluation  policies,  encloses  milestone  checklists  tor  1X1  planning,  and  implements  the 
policies  of  higher  authority  through  the  lest  and  Evaluation  Master  Plan  ( II  MIM  (direc- 
tions for  preparing  the  TEMP  are  included).  OPNAVINS1  5440.47  is  useful  in  understand- 
ing the  functions  of  COMOPTI  VFOR  The  3960  series  instructions  provide  considerable 
guidance  in  the  establishment,  planning,  and  implementation  of  l&E  programs;  OPN  \\  INS  I 
3960.10  should  be  followed  in  requesting  formal  status  to  obtain  ('OMC)Pll  \ fOK  assis- 
tance and  to  request  fleet  services.  The  following  portions  of  this  chapter  are  intended  to 
amplify  these  instructions  and  to  provide  more  detailed  guidance  in  structuring  the  I I Ml 


GENERAL  PLANNING  FOR  EACH  PHASI 

When  program  planning  is  initiated.  T&h  plans  should  be  an  integral  part  d the 
program  plans.  There  are  three  types  of  plans  for  T&l  the  II  MP.  the  master  program 
test  plan,  and  the  detailed  test  plan.  The  IT  MP  should  be  established  as  early  as  possible 
since  it  is  the  planning  summary  which  serves  as  a contact  with  the  "outside  world 
The  program  l&l  plan  should  always  be  more  detailed  than  the  II  MP  ilselt  I In  mas  t 
program  T&E  plan  should  initially  outline  the  purposes,  scope,  and  objectives  of  the  I I 
intended  for  each  program  phase.  Referring  to  table  XI V I testing  occurs  at  several 
levels  of  complexity  in  each  phase;  it  is  useful  to  structure  the  master  program  test 
plan  to  conform  to  the  Work  Breakdown  Structure  (WBSt  (see  chaptei  111  Program  Plan- 
ning) so  the  individual  test  sequences  can  be  scheduled,  costed  out  cooid  naled.  and 
tracked  at  each  level.  The  master  program  test  plan  will  include  tin  schedules  tot  request 
mg  services,  for  preparing  test  facilities,  for  submitting  and  updatm  the  II  MP  an  I lot 
executing  detailed  test  plans  existing  tor  each  cell  ot  the  WBS  additionallv  tin  scope  and 
objectives  will  be  continually  updated  within  the  master  program  test  plan  1 he  detailed 
test  plans  are  formulated  to  conform  to  the  master  plan  prior  to  the  initiation  ol  the  test 
phase  to  which  they  apply;  they  contain  the  characteristics  to  be  tested  lor  the  test  pro 
cedures  to  be  followed;  a description  of  the  test  setups,  lacilities.  and  test  equipments 
needed;  and  the  documentation  to  be  generated  in  support  ot  the  I XI  ettort 


The  TEMP  and  the  master  program  test  plan  should  be  similar  in  content  and  dif- 
ferent primarily  in  format.  Generally,  the  master  program  test  plan  will  be  extended  to  a 
lower  level  of  the  WBS  than  the  TEMP.  It  will  also  contain  cost  estimates  for  each 
detailed  test  and  will  list  the  alternative  plans  supporting  backup  approaches.  Both  docu- 
ments are  the  responsibility  of  the  government  project  manager. 

The  detailed  test  plans  are  normally  written  by  the  executing  activity,  whether 
in-house  or  contractor,  and  submitted  to  the  government  project  manager  for  approval. 

The  detailed  tests  respond  to  the  applicable  portions  of  system  specification(s).  To  check 
the  accuracy  of  a detailed  test  plan,  the  plan  is  compared  to  the  section  4 provisions  of 
the  specification,  which  should  contain  all  the  inspections  and  tests  to  be  performed  to 
ensure  conformanee  to  the  specification  requirements  (section  3).  The  Documentation 
section  (chapter  XX)  contains  a list  of  suggested  data  item  descriptions  (l)IDsj  to  be  used 
in  requiring  test  plans  and  test  reports  in  a contract  data  requirements  list  (CURL).  A 
test  report  is  comprised  ol  the  detailed  test  plan  plus  test  results  consisting  of  a summa- 
tion and  analysis  and  of  test  data.  Most  formal  test  reports  are  prepared  in  accordance 
with  MIL-S TD-831. 

The  specifications  quality  assurance/conformance  provisions  must  detail  the  follow- 
ing information: 

Parameters  and  attributes  to  be  confirmed;  accept. /reject  criteria 
En vii onmen tal  conditions 
Environmental  stress  levels 
Extent  of  testing 

Standard  test  methods  and  procedures  to  be  employed 

Data,  analyses,  and  reports  required  to  demonstrate  conformance 


lest  requirements  originating  from  other  sources  such  as  IES  plans  must  provide  this  same 
information.  Table  XIX-2  shows  typical  environmental  test  requirements.  The  parameters 
and  attributes  should  include  all  the  form.  fit.  and  function  characteristics  which  are  required, 
the  accept/reject  criteria  should  be  based  upon  the  specified  parameter  tolerances  and  clearly 
defined  attributes.  The  accept/reject  criteria  will  normally  change  as  a function  of  environ- 
mental stress  level;  the  stress  levels  are: 


Operational 
Simulated  Operational 
Overstressed 

Uncontrolled 
Combined  Environment 


The  actual  application  environment 
Laboratory -controlled  environment 

Allowed  excursions  beyond  the  specified  normal 
conditions 

Laboratory  bench/ambient  environment 
Simulated  multiple  environments 


Most  preliminary  evaluation  testing  is  performed  in  an  uncontrolled  environment;  final 
evaluation  testing,  acceptance  testing,  and  verification  testing  utilize  simulated  operational 
environmental  levels.  Simulated  environments  may  be  tested  singly,  doubly,  or  in  multiple 
Single  simulations  are  normally  used  because  diagnosis  test  data  may  be  simultaneously 
extracted  and  readily  analyzed  to  isolate  the  origins  of  undesirable  behaviors  and  the  causes 
of  test  failures.  Double  simulations  are  used  when  two  environmental  factors  are  very 
closely  related,  such  as  temperature  and  humidity  . 1 lie  combined  environment  test  (Cl  I ) 
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TABLE  \l\-2.  EQUIPMENT  1 NV1RONM1  NTAl.  IT  STS  AND  REQUIRI  Ml  MS. 


GI  NK RAL  REQU I REM l NTS 


Designator 


Environment 


I 


4 

5 

6 

7 

8 
0 
10 
1 1 
12 
IT 

14 

15 
10 
17 
IS 
10 
20 
21 


24 

25 

2o 

T7 

2S 
20 
TO 
T 1 

42 

TT 

T4 

T5 

TO 

T7 


Temperature  (operating  and  nonoperating) 

Temperature -alt  it  ude 

Humidity 

Thermal  shock 

Vibration 

High  impact  shock 

Transport  shock 

Repetitive  shock 

EMC  (interference  and  susceptibility) 

EMP 

Electrical  transients  (voltage  and  frequence /long  term  and  short  term) 

Lightning 
Magnetic  field 

Acoustic  noise  (airborne  and  structureborne) 

Inclination 
Radiation 
Nuclear  air  blast 
Gun  blast 
Wind 

Icing  with  wind 

Rain  and  snow,  snow  loading 

Sunshine 

Degree  of  enclosure 
Dust 

Stilt  fog,  spray,  solution 
Damaging  (corrosive)  atmospheres 
Explosive  atmosphere 
Fungus 

Maintainability  bench  handling 

Reliability  (burn-in.  confidence,  indexing,  accelerated  life,  failure-mode  analysis) 
Combined-environment  testing  ( tempera ture -humid it y -vibration -elect ucal  traiisients- 
on/off  cycling) 

On/off  cycling 

Acoustic  susceptibility  (in  high-noise  environments) 

Water  impact/hydrostatic  pressure 

Underwater  explosion  (for  hull-mounted  equipments  only ) 

Drop  test 

Equipment  special  environments 


( ategors 

Vital  equipments 

Semivit.il  equipments 
Nonvital  equipments 
Exposed  equipments 
Sheltered  equipments 
Standard  requirements 


SPECIAL  REQUIRI  Ml  NT  CATEGORIES 


I rtvironmetils 


Notes 


It).  Id.  I7 
0 


(i 

0 

12.18,  |>>.  20,  21 . 


24 


I . T.  5.ts.<).  1 1 . 14. 2<>,  TO,  T I . T2.  T7 


l(i  and  17  for  exposed  equipments 
operating 

nonoperating  (normal  opeialing  hetoie  and  altei  slnx.  ki 
safely  criteria 
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1 1 *1  nvimnmental  requirements  should  consider  standard,  sheltered  or  exposed,  and  vital-seimvital-nonvital  requirements  in  addition  to  those  listed 


i.s  a multiple  simulation  intended  to  elosely  resemble  the  actual  operational  environmental 
extremes.  Temperature,  humidity,  vibration,  electrical  transients  (voltage  and  frequency), 
and  K I I environments  can  be  economically  tested  together:  since  these  environments  may 
produce  synergistic  effects  in  combination  which  are  not  detectable  in  single-environment 
tests.  (TT  is  an  essential  tool  for  preliminary  qualification  testing  (as  for  the  TELCAM 
screen)  and  for  developing  confidence  that  equipments  will  operate  satisfactorily  in  the 
operational  environments  (ref  NELC  Technical  Report  1605).  (TT  is  not  normally  appli- 
cable for  test  phases  employing  diagnosis  because  fault  data  cannot  be  easily  traced  to  the 
environmentally  susceptible  portion  of  the  design:  therefore.  (TT  is  normally  preceded  by 
sequential  single-environment  testing  which  will  often  utilize  overstress  parameters.  When 
synergistic  faults  do  occur,  a multiple  linear  regression  model  can  be  employed  to  break 
down  the  environmental  performance  of  the  equipment;  since  these  models  require  many 
data  points  and  computer  support,  it  is  usually  more  economical  to  rely,  at  least  initially, 
on  previous  single-environment  results  to  try  to  discover  weak  design  points.  All  final 
qualification  testing  should  utilize  an  operational  environment.  In  addition,  evaluation 
testing  (especially  of  ADMs)  of  equipments  which  have  critical  interfaces  (either  opera- 
tionally or  functionally  ) with  the  ultimate  platform  should  be  performed  in  the  operational 
environment;  table  XIX-3  illustrates  the  different  fleet  services  which  may  be  requested 
for  such  testing.  Requests  must  be  made  in  accordance  with  Ol’NAV  INST  3960. 1 (). 


TABLE  XIX-3.  FLEET  SI  RVICES  WHICH  MAY  BE  REQUESTED 
THROUGH  COMOPTEVFOR  TO  SUPPORT  TEST  PROGRAMS. 

Elect  Research  Investigation  An  examination  of  natural  or  special  phenomena  in  an  opera- 

tional environment,  required  by  a development  agency  (HA)  in 
the  prosecution  of  research  and  tor  which  the  assistance  ol 
operating  forces  is  required.  The  reseatch  investigation  is  not 
necessarily  program  oriented,  but  is  primarily  in  the  pursuit  of 
basic  research  to  provide  a continuing  data  base  which  may 
have  potential  application  in  areas  of  naval  interest.  I lie  DA  is 
responsible  lor  the  planning  of  Fleet  Research  Investigations, 
and  further  advises  the  operational  commander  providing  the 
support.  COMOPTEVFOR  shall  make  arrangements  for  needed 
fleet  support,  as  requested  by  the  DA. 

Development  Assist  Projects  to  provide  tleet  support  for  the  test  needed  in  gathering 

data  to  determine  the  direction  in  which  development  should 
proceed,  in  response  to  a requirement  during  the  conceptual 
phase  of  a program.  They  may  also  relate  to  material  improve- 
ments of  equipments  already  in  the  tleet.  (Examples  proposed 
SH1PALTS.  ORDALTS.  Service  changes.) 

Several  such  projects  may  be  requested  to  provide  sufficient 
data  tor  (he  preparation  of  a IX'P  or  comparable  document 
The  Development  Assist  tests  are  primarily  the  responsibility 
ot  the  DA  who  plans  the  tests.  The  test  plan  promulgated  by 
the  DA  will  be  mutually  agreeable  with  COMOPTEVFOR  and 
the  DA  with  respect  to  fleet  unit  participation. 

Operational  Assist  An  Operational  Assist  protect  is  assigned  by  ( NO  in  response  to 

a favorable  program  initiation  decision  by  SI  ( DTI  or  compaia 
hie  ( NO  decision.  In  addition,  a DA  ma\  request  an  Operational 
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TABLE  XIX-3.  (Continued!. 


Operational  Assist 
(Continued! 


Technical  Evaluation  Project 
(TECHEVAL) 


C Operational  Evaluation 


Assist  project  for  material  improvement  programs  and  for  cer- 
tain acquisition  programs  by  submission  of  a project  request. 

This  project  is  primarily  the  responsibility  of  the  DA,  and  its 
major  purpose  is  to  establish  confidence  in  the  program  worth 
and  readiness  for  the  commitment  of  resources  for  full-scale 
development. 

A TECHEVAL  is  assigned  by  CNO  in  response  to  a favorable 
program  full-scale  development  decision  by  SI  (1)1  I or  com- 
parable CNO/CNM  decision  approving  the  commitment  of 
resources  for  full-scale  development.  In  addition,  a Tl  till  VA1 
may  be  requested  by  a DA  to  determine  suitability  lor  other 
acquisition  programs  and  material  improvement  including  con- 
versions, major  modifications,  and  modernizations.  In  the  case 
of  aircraft  or  missile  programs,  the  TECHEVAL  will  include  the 
Navy  Preliminary  Evaluation  (MM  I or  the  Navy  Technical 
Evaluation  (NTE  ). 

A TECHEVAL  is  performed  for  the  purpose  ol  investigating 
systems  or  equipments  and  collecting  information  which  will 
aid  in  answering  technical  questions  and  issues.  The  testing 
and  analysis  conducted  by  or  for  the  DA  during  the  tune  span 
of  the  TECHEVAL  project  are  to  permit  the  DA  to  determine 
whether  the  system  or  equipment  is  functioning  in  a technically 
acceptable  manner,  meets  design  and  technical  performance 
specifications,  and  is  technically  suitable  for  Operational  Evalu- 
ation (OPEVAL). 

The  DA  has  primary  responsibility  for  planning  the  test  program, 
including  the  coordinated  operational  inputs  ol  COMOPTI  VEOR 
TECHEVAL  and  OPEVAL  are  complementary  test  programs  and 
complete  initial  operational  test  and  evaluation.  Together  they 
generate  data  and  address  the  spectrum  ol  questions  and  issues 
to  be  considered  prior  to  a major  production  decision.  Accord- 
ingly, through  close  liaison,  COMOPTE  VEOR  and  the  DA  shall 
ensure  that  the  test  plans  are  mutually  agreeable  and  integrated 
to  the  extent  that  they  adequately  address  the  critical  questions 
and  issues  posed  in  the  governing  IM  P or  comparable  document, 
and  provide  for  the  maximum  practicable  use  of  common  test 
data. 

Testing  during  TECHEVAI  mas  be  conducted  using  production 
prototype  or  pilot  production  models.  Prior  to  OP1  VAI  . the 
DA  shall  institute  a design  freeze  on  the  equipment  or  system 
and  certify  it  to  CNO  COMOPTI  VEOR  as  ready  foi  DPI  \ \l 

At  the  time  the  TECHEVAI  is  assigned  by  ( NO.  an  OP  I \ \l 
will  also  be  assigned,  primarily  tor  plarin mg  and  scheduling 
purposes.  Project  operations  under  OPI  VAI  will  not  commeiue 
until  the  DA  has  certified  the  equipment.  The  protect  is  the 
responsibility  ol  COMOPTI  VEOR  with  the  development  agency 
assisting  as  required.  COMOPTI  VI  OR  collects  data  ot  reasonable 
scope  and  depth  to  aid  in  the  decision  making  pros  ess  at  ptogtum 
milestones. 
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TABLE  X1X-3.  (Continued). 

Operational  Evaluation  The  primary  objectives  of  OPEVAL  arc  to  ascertain  that: 

(Continued)  Xlie  system  or  equipment  functions  in  an  operationally  satis- 

factory manner  and  performs  reliably  and  effectively  in  accord- 
ance with  program  objectives  in  realistic  operational  conditions. 

The  system  can  be  effectively  operated  and  maintained  by 
the  level  of  personnel  skill  anticipated  to  be  available  under 
service  conditions. 

There  is  reasonable  indication  that  logistic  supportability  in 
a deployed  status  is  feasible. 

All  test  questions  germane  to  a production  decision  are 
adequately  examined. 

Fleet  Operational  Appraisal  The  assignment  of  a Fleet  Operational  Appraisal  project  by 

CNO  may  be  initiated  by  a recommendation  submitted  by 
COMOPTEVFOR  on  completion  of  an  OPEVAL  or  by  request 
of  a Fleet  Commander  in  Chief  or  type  commander.  The 
purpose  of  this  type  project  normally  is  to  provide  for  follow- 
on  operational  test  and  evaluation,  to  develop  optimum  proce- 
dures and  tactics  for  existent  systems  or  equipment  both  new 
and  existent  in  the  fleet,  to  verify  the  performance  of  produc- 
tion units,  and/or  to  confirm  the  correction  ol  discrepancies 
previously  disclosed.  It  also  may  be  used  to  examine  and/or 
develop  concepts  and  procedures  to  better  define  and  determine 
requirements  for  future  systems  development. 

The  confidence  attainable  through  testing  relies  on  four  factors: 

Accurate  specifications  of  the  test  parameters  and  the  environmental  conditions 
Accept/reject  criteria 
Extent  of  testing 
Test  methodology 

The  accuracy  of  the  specification  is  primarily  a function  of  the  information  available  to 
put  into  it.  Good  records  of  prior  work  on  the  program,  good  test  data  practices,  research 
of  related  investigations  (see  chapter  IV.  Conceptual  Phase,  Gathering  Information),  and 
previous  experience  with  similar  existing  equipments  (see  chapter  IV.  Gathering  Inhuma- 
tion, and  chapter  VII.  Initiating  Support.  Monitoring  the  Performance  of  Equipments  in 
the  Fleet)  will  contribute  to  accurate  parameter  specifications;  table  X1X-2  and  appendix 
A are  provided  to  more  precisely  determine  environmental  conditions.  I he  accept  reject 
criteria  can  be  manipulated  with  respect  to  allowed  parameter  tolerances  lo  yield  higher 
confidence  results.  As  the  environmental  stress  level  is  eased  trom  operational  to  simulated 
to  uncontrolled,  the  accept/reject  criteria  should  normally  be  tightened  with  respect  to  Un- 
specified tolerances  to  compensate  for  the  lost  test  accuracy.  While  common  sense  and 
knowledge  of  the  design  may  temper  the  final  determinations,  a rule-of-thumb  is  to  tighten 
the  criteria  by  10'V  lor  each  easing  ol  stress  level  and  by  5h  lor  each  lesl  phase  preceding 
NTE/OPEVAL.  The  extent  of  testing  involves  the  length  of  testing  and  the  number  ol 
units  subjected  to  test;  higher  confidences  may  be  obtained  with  more  test  time  and  more 
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guidance  on  the  test  time  and  equipment  quantities  which  are  required  to  achieve  various 
levels  of  confidence.  The  standard  test  methodologies  should  he  utilized  wherever  applicable 
as  modified  by  the  application  environment,  since  standard  test  procedures,  test  equipment, 
and  test  facilities  are  available  and  a wealth  of  experience  with  the  methodologies  has  been 
developed  over  the  years;  table  XIX-4  lists  the  more  important  test  method  standards. 
Parameters  which  lend  themselves  to  statistical  analysis  may  be  tested  satisfactorily  with  a 
sampling  procedure;  others  dependent  on  nonvarying  design  features  require  the  testing  of 
all  test  items.  Each  parameter  should  be  reviewed  with  sampling  in  mind  when  large  quan- 
tities of  test  items  arc  involved,  since  test  costs  can  be  significantly  reduced  by  sampling 
procedures.  Different  numbers  of  test  items  may  be  used  for  various  tests;  generally,  the 
test  item  population  will  be  divided  up,  with  some  tests  performed  on  the  entire  popula- 
tion and  different  tests  run  on  the  various  portions  of  the  population. 


TABLE  XIX-4.  TEST  METHODOLOGY  STANDARDS  * 


M1L-STD-105 

MIL-STD-108 


MIL-STD-167 
MIL-STD-1 70 
M1L-STD-220 
MIL-STD-271 
M1L-STD-414 
M1L-STD-446 
MlL-STD-449 
MIL-STD-462 
MIL-STD-469 
MIL- STD-471 
M1L-STD-690 
MIL-STD-740 

MIL-STD-750 
Mil -STD- 78 1 
Ml!  STD-810 
Mil -STD-88,'. 
MIL  S I D- 1210 

Mil  STD-121 1 
Mil  STD-1244 
Mil  SID-1472 
Mil  S-l>01 

Mil  It  2087 
Mil  I-S422 
Mil  D-l  .270 


Sampling  Procedures  and  Table  for  Inspection  by  Attributes 

Definitions  of  and  B,  ic  Requirements  for  Enclosures  for  Electric  and  Electronic 

Equipment 

Mechanical  Vibrations  of  Shipboard  Equipment 
Moisture  Resistance  Test  Cycle  for  Ground  Signal  Equipment 
Method  of  Insertion-loss  Measurements 
Nondestructive  Testing  Requirements  for  Metals 

Sampling  Procedures  and  Tables  for  Inspection  by  Variable  for  Percent  Detective 

Environmental  Requirements  for  Electronic  Parts 

Radio  Frequency  Spectrum  Characteristics.  Measurement  of 

Electromagnetic  Interference  Characteristics,  Measurement  of 

Radar  Engineering  Design  Requirements  for  Electromagnetic  Compatibility 

Maintainability  Demonstration 

Failure  Rate  Sampling  Plans  and  Procedures 

Airborne  and  Structureborne  Noise  Measurements  and  Acceptance  Criteria  ol 

Shipboard  Equipment 

Test  Methods  for  Semiconductor  Devices 

Reliability  Tests.  Exponential  Distribution 

Environmental  Test  Methods 

Test  Methods  and  Procedures  for  Microelectronics 

Shipboard  Bonding,  Grounding,  and  Other  Techniques  lot  Electromagnetic  Com- 
patibility 

Test  Methods  for  Electron  Tubes 
Test  Methods  for  Electrical  Connectors 

Human  Engineering  Design  Criteria  for  Military  Systems,  Equipment,  and  Facilities 
Shock  Tests.  High  Impact;  Shipboard  Machinery,  Equipment,  and  Systems, 
Requirements  for 

Bonding,  Electrical,  and  Lightning  Protection  for  Verospace  S\  stems 
Testing,  Environmental.  Airborne  Electronic  and  Associated  I quipment 
Dust,  festing  by  Exposure  to 


‘Additional  methods  are  specified  by  the  various  general  equipment  specifications  (Mil  I I (.400  Mil 
1-28800.  Mil  I 2400,  etc)  and  b\  military,  federal,  and  industrial  standards.  Reter  to  the  Dol)  Index 
ol  Specifications  and  Standards  for  methods  of  less  general  application  and  tor  the  most  recent  editions 
o|  the  above  standards. 
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Overstievsed  environments  may  .ipply  n either  operational  or  simulated  environ- 
mental test  levels  When  present  n operational  rniionments  the  eondlt lolls  of  overstress 

and  allowable  deeradution  oi  ih  pn| 1 • ■ •. i Ul  Is  m hided  in  the  specilieation.  Cher- 

stressed  conditions  may  aUo  I . api'h  >1  1 an  dat  vl  . nvitotimerits  to  i I | accelerate  aping 
factors.  (2)  discovei  environment. illv  ptibl  ,l,sii’i  i attics  and  ( I increase  test 

confidence  While  vci>  nsetiil.  tit  ti.  i i.  should  Ih  understood  .is  to  how  they 

should  affect  a design  so  that  icalMi  i nt.  ria  mav  be  formulated.  The  nature 

of  some  test  items  precludes  v rt  .in  tvp  ’-ii.-s  (paper  tape  cannot  be  tested  at 

100  humidity  lot  instance)  Win  i th  i - hanisms  arc  not  know n.  overstresses 
not  reflected  ill  the  iisagi  environment  .bo  U •’  iv.uded 

I SUM  \ 1 1 Nt . t OSIN  OI  IM 

Estimating  T&F  costs  is  an  important  li  tion  to  program  planning  (see  chapter 
III),  Farly  detailed  f&l  master  planning  an  help  significantly  nevertheless,  there  are 
many  uncertainties  for  which  contingency  plans  must  be  budgeted.  I he  following  list  of 
factors  should  be  considered  in  I .VI  cost  estimating 

Facilities  (ships,  test  beds,  laboratoric" 

Test  equipments 

Instrumentation 

Documentation  (detailed  test  procedures,  technical  manuals  analyses,  reports. 

equipment  surveys,  etc) 

Personnel  (types,  time  needed) 

Repair  capabilities  (including  technicians,  technical  manuals,  test  equipments. 

tools,  spare  parts) 

Failures  will  occur  during  testing;  depending  on  the  design  and  complexity  ot  the  equip- 
ment. the  repair  functions  can  double  or  triple  the  T&F  period  and  increase  costs  by  a 
factor  of  2 to  10  (5  is  considered  a good  planning  figure).  A failure  to  meet  the  test 
objectives  is  much  more  probable  in  early  program  phases  than  later  if  the  test  planning 
the  entire  program  planning,  for  that  matter  is  done  properly.  Therefore.  F&l  contin- 
gencies should  be  made  larger  m the  early  phases  relative  to  the  estimated  “no  failure" 
cost  than  in  later  phases.  I he  T&F  hedules  should  also  reflect  contingencies  and  possible 
slippages,  and  the  program  plan  should  incorporate  the  necessary  flexibility  to  overcome 
these  uncertainties.  T&F  personnel  (such  as  NFLC'  Code  4700)  should  be  consulted  in 
structuring  the  T&F  portions  of  the  program  plan,  in  formulating  the  various  test  docu- 
ments, and  in  scheduling  their  environmental  and  simulation  facilities 

T&F  PLANNING 

It  is  essential  that  sufficient  developmental  and  operational  test  and  evaluation  be 
accomplished  to  ensure  that  the  systems  introduced  into  service  meet  the  defined  opera- 
tional requirements  and  are  suitable  for  service  use.  The  operational  effectiveness  and 
suitability  of  these  systems  must  be  determined  to  a high  degree  of  confidence  before  they 
are  approved.  Complete  and  realistic  programs  tor  such  determinations  are.  therefore. 

matters  of  concern  to  all  levels  of  management.  \ 
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The  Commander.  Operational  Test  and  Evaluation  Force  (COMOPTEVFOR).  has 
been  assigned  responsibilities  as  an  independent  test  agency  for  the  required  operational 
test  and  evaluation  of  new  systems.  The  objectives  of  COMOPTEVFOR  are  to  ensure  that 
early  and  realistic'  operational  tests  and  evaluations  are  planned;  to  ensure  that  appropriate 
initial  operational  tests  and  evaluations  are  conducted  prior  to  scheduled  decision  milestones; 
and  to  provide  an  independent  assessment  of  the  operational  effectiveness  and  suitability  of 
the  system  to  CNO. 

CNO  has  responsibility  for  ensuring  the  adequacy  of  the  Navy’s  overall  test  and 
evaluation  program.  T&E  policy  and  guidance  are  exercised  through  the  Director,  RDT&E 
(OP-098),  and  ;n  accordance  with  overall  policies  established  by  the  Secretary  ol  the  Navy. 
The  Director';  sponsibilities  include: 

Reviewing  proposed  T&E  objectives  and  requirements  for  fleet  services  to  support 

T&E  programs. 

Assigning  all  T&E  projects  and  their  priorities  to  the  operating  forces  for  prosecution. 

Receiving,  reviewing,  and  assessing  all  OT&E  reports  lor  systems,  and  acting  lor 

CNO.  as  the  sole  releasing  authority,  for  such  reports  and  information  to  higher 

authority. 

Ensuring  that  adequate  funding  for  all  required  and  approved  T&E.  is  identified  in 

the  programming  and  budgetary  system. 

TEST  AND  EVALUATION  OF  MAJOR  SYSTEMS 

rest  and  evaluation  of  systems  is  discussed  in  the  following  three  subsections. 

DEVELOPMENTAL  TEST  AND  EVALUATION  ( DT&E ) 

Phis  category  or  class  of  tests  and  evaluations  is  conducted  under  (lie  sponsorship 
of  the  developing  agency,  and  is  undertaken  tor  the  specific  purpose  of  facilitating  the 
evolution  of  a system.  The  development  test  plan  must  be  structured  and  executed  in  such 
a manner  as  to  ensure  the  generation  of  data  which  can  also  be  used  to  make  the  required 
assessments  of  the  operational  effectiveness  and  suitability  ol  the  emerging  design,  lest 
programs  in  this  category  include  engineering  tests,  contractor/laboratory  demonstrations, 
naval  technical  evaluations,  and  Navy  preliminary  evaluations  and  deficiency  correction  tests, 
fhe  test  articles  required  by  the  DT&I  programs  are  characterized  as  advanced  development, 
engineering,  and  production  prototype  models. 

OPI  RATIONAL  TEST  AND  E VALUATION  (OT&E  > 

l lus  category  of  testing  includes  all  test  and  evaluation  eltort  participated  in.  or 
undertaken,  for  the  purpose  of  obtaining  operational  information  throughout  the  life  cycle 
of  the  system,  and  supports  both  the  acquisition  process  and  the  optimum  employment  ot 
the  system.  Il  is  accomplished  in  two  phases.  Initial  Operational  iest  and  Evaluation 
1 1 O'  I & 1 t and  follow-on  Operational  lest  and  Evaluation  (FOT&E). 
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IOT&F  is  that  testing  and  evaluation  accomplished  by  or  under  the  supervision  of 
the  Navy’s  independent  testing  agency  prior  to  the  first  major  production  decision. 
lOT&E  reports  address  the  operational  effectiveness  and  suitability  (including  reliability, 
compatibility,  and  maintainability)  of  the  system,  including  the  critical  issues  identified  in 
the  DC'P  or  comparable  acquisition  document  in  accordance  with  defined  operational  re- 
quirements. lOT&E  is  conducted  in  accordance  with  DCP. 

FOT&E  is  the  continuing  test  and  evaluation  of  a system  conducted  under  fleet 
conditions  by  fleet  operational  personnel  using  initial  production  systems  for  the  purposes 
of  verifying  system  performance,  validating  correction  of  deficiencies  previously  identified, 
and  refining  tactical  employment  doctrine  and  requirements  for  personnel  and  training. 


ACCEPTANCE  TRIALS 


The  Board  of  Inspection  and  Survey  (BIS)  is  responsible 'to  CNO  for  conducting 
acceptance  trials  prior  to  Navy  acceptance  from  the  contractor.  The  Board  inspects  the 
material  condition,  requires  demonstration  of  equipments  and  systems  to  ensure  performance 
in  accordance  with  the  requirements  of  the  contract  specifications,  and  determines  com- 
pliance with  the  characteristics  established  by  CNO.  After  completion  of  acceptance  trials 
the  Board  reports  its  findings  to  CNO.  identifying  deficiencies  for  which  they  recommend 
corrective  action  prior  to  delivery.  Upon  successful  completion  of  final  contract  trials, 
the  Board  reports  to  CNO  its  recommendations  for  final  settlement  of  the  contract. 
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The  usual  sequence  of  test  events  followed  during  the  acquisition  of  a system  is 
illustrated  in  figure  XIX-1.  Those  major  programs  subject  to  DSARC  review  shall  adhere 
to  the  following  procedures. 

During  the  drafting  of  the  initial  DCP  the  CNO  development  and  program  coordin- 
ators shall  coordinate  with  the  CHNAVMAT  or  development  agency  project  manager  and 
COMOPTEVFOR  in  preparing  the  statement  of  critical  questions  and  issues  which  tests 
will  address.  Plans  and  estimates  of  the  required  test  and  evaluation  shall  be  incorporated 
and  schedule  milestones  identified.  The  proposed  test  and  evaluation  programs  and 
schedules  will  be  reviewed  for  compliance  with  Navy  policy  by  CNO  (OP-‘)83). 

During  the  advanced  development  or  validation  phase,  subsequent  to  DSARC  I 
the  development  agency  and  OPTEVFOR  will  refine  the  critical  questions  and  issues  to 
be  addressed  by  T&E.  Prior  to  DSARC  II  COMOPT1  VFOR  will  make  an  assessment  ol 
the  operational  suitability  and  effectiveness,  based  upon  all  valid  IOT&F  data  extracted. 
The  applicable  portions  of  the  DCP  for  DSARC  II  will  be  updated  with  tins  additional 
information  by  the  C NO  development  and  program  coordinators.  The  refined  critical 
questions  and  issues  and  the  test  program  will  be  reviewed  for  adequacy  and  compliance 
with  Navy  T&F  policy  by  CNO  (OP-1) S3 ).  The  revised  DCP  will  reflect  the  status  ol  devel- 
opmental testing  and  indicate  technical  and  operational  risks  outstanding 

Once  full-scale  development  has  been  authorized  by  SF.CDFI  after  DSARC  mile- 
stone 11.  DT&E  will  be  oriented  toward  resolving  the  applicable  critical  questions  and 
issues  and  achieving  a system  that  is.  in  all  respects,  ready  for  production  and  suitable  for 
service  use.  Prior  to  DSARC  milestone  III  lOT&E  will  address  operationally  oriented 
critical  questions  and  issues,  using  data  generated  in  the  DT&I  programs.  BIS  trials,  and 
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Figure  XIX-!.  Sequence  of  acquisition  test  events. 
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ENGINEERING  TESTS 


operational  tests.  OPTHVFOR  will  gather  all  valid  IOT&1  results,  evaluate  them,  and  pre- 
pare an  independent  assessment  of  system  operational  effectiveness  and  suitability.  This 
report  will  be  directly  forwarded  to  (’NO.  CNO  will  provide  an  assessment  of  the  test 
results  in  terms  of  the  response  to  the  critical  questions  and  issues  to  the  appropriate  OSD 
offices. 

Follow-on  test  and  evaluation  will  be  conducted,  following  a favorable  production 
decision  as  directed  by  CNO.  Follow-on  OT&F  may  be  desirable  in  order  to  verify  per- 
formance of  the  new  production  units,  further  develop  doctrine  and  tactics,  and  complete 
the  evaluation  ot  the  reliability,  maintainability,  and  logistic  supportability  of  the  produc- 
tion system. 

1NSURV  conducts  acceptance  for  service  use  and  specifies  conditions  for  accept- 
ance from  the  contractor.  Acceptance  for  service  use  is  required  prior  to  delivery  by  the 
contractor.  The  operational  tests  and  evaluations  by  OPTEVFOR  and  the  acceptance  for 
service  use  by  1NSURV  are  mutually  supporting.  An  integrated  test  program  is  desirable 
to  prevent  duplication  and  to  make  available  to  both  OPTFVFOR  and  1NSURV  all  lest 
data  as  they  become  available. 


T&E  MANAGEMENT  OF  OTHER  THAN  MAJOR  SYSTEMS 

Programs  of  this  category  are  not  subject  to  the  DSARC  decision  processes.  They 
shall  be  subject  to  a similar  review  and  decision  process  within  the  Department  of  the 
Navy.  ( NO  monitors  these  programs  and  renders  the  productive  decision  after  appropriate 
developmental  and  operational  test  and  evaluation  of  the  new  system  lias  been  accomplished. 
(’NO  will  delegate  the  review  process  to  CHNAVMAT.  Normally.  ( NO  will  exercise  the 
review  and  decision  process  for  those  programs  having  an  RDT&F  cost  in  excess  of  S5 
million,  or  a production  cost  in  excess  of  $20  million.  However.  ( NO  may  exercise  the 
review  and  decision  process  for  programs  below  these  thresholds  dependent  upon  the 
mission  essentiality  of  the  program.  The  test  and  evaluation  requirements  shall  parallel 
those  established  for  major  programs. 


T&L  MANAGEMENT  OF  SHIP  CONSTRUCTION 

In  the  systems  acquisition  process,  ship  construction  is  a unique  procedure.  The 
test  and  evaluation  for  major  or  other  new  systems  in  a ship  shall  be  accomplished  with 
the  cognizant  program.  Appropriate  information  will  be  provided  in  the  Ship  Program 
DCP  on  which  to  base  a ship  production  decision  and  schedule  of  specific  follow-on  manage- 
ment reviews  for  the  ship  program. 

The  requirement  to  test  and  evaluate  a system  prior  to  production  approval  may 
only  be  waived  by  the  Secretary  of  the  Navy. 

Major  programs  require  SI  ('DI  F approval. 


MTI  I \R  Wl  VI’ONS  TAP 

limit  \l(  l)ol>  development  of  a new  nuclear  weapon  is  governed  by  the  l'>5$ 

\ it  between  the  AFC  and  the  Dol)  for  the  Development.  Production  and  St.uul- 

t \Ioiiik  Weapons."  lest,  evaluation,  and  acceptance  of  nuclear  weapons  are 
i d in  .iiiord.mce  with  that  agreement  and  implementing  Dol)  issuances  and 
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XX.  DOCUMENTATION 

PM  GUIDELINES  . . . page  XX- 1 
TYPICAL  DATA  ITEMS  . . . XX-3 

System/Equipment  Description  . . . XX-9 
Design  Change  Control  . . . XX- 10 
Support  Documentation  . . . XX- 12 


XX.  DOCUMENTATION 


The  goal  of  the  government  with  respect  to  documentation  lias  always  been  to 
acquire  only  the  data  essential  for  adequate  development,  procurement,  operation,  and 
support  of  systems  and  equipment.  Yet  the  volume  and  cost  of  data  have  somehow  grown 
over  the  years  until  today  both  are  enormous.  DoD  is  therefore  obliged  in  the  interest  of 
meeting  its  goal  to  fight  a constant  holding  battle  against  proliferation  of  data  require- 
ments. Directive  after  directive  demands  the  exercise  of  judgment  and  restraint  in  the 
specification  of  data  items  for  procurement. 

The  problem  at  the  procuring  level  lies  in  determining  the  essential  minimum, 
which  is  by  no  means  necessarily  the  absolute  minimum.  Underordering  can  be  as  costly 
as  overordering.  The  manager  who  habitually  buys  only  contractor  drawings  is  often  one 
day  obliged  to  pay  sky-high  prices  for  data  which  must  be  ordered  on  unfavorable  terms 
after  the  original  contract  is  let.  (Furthermore,  once  pinched  by  his  own  frugality,  such 
a manager  is  likely  to  err  in  the  opposite  direction  to  routinely  order  all  the  documen- 
tation for  which  a need  may  conceivably  develop.) 

The  answer  to  the  problem  lies  in  management.  The  project  manager  must  define 
data  requirements  as  rigorously  as  equipment  requirements,  especially  when  the  emphasis 
is  on  low-cost  acquisition. 


PM  GUIDELINES 

The  project  manager  can  simplify  the  documentation  task  by  obtaining  assistance 
from  data  management  specialists  (Product  Assurance  Department,  at  NOS(').  by  procuring 
documentation  through  the  internal  organizations  established  for  that  purpose  (Technical 
Information  Division,  at  NOSC),  and  by  observing  the  following  key  guidelines: 

1.  On  the  basis  of  project  requirements  and  the  end  use  of  the  related  system/ 
equipment,  and  as  early  as  possible,  establish  a preliminary  shopping  list  of 
the  types  of  documents  that  you,  as  the  developer,  and  the  government,  as 
the  user,  conceivably  could  need  to  permit  effective  decisions,  evaluations, 
and  actions.  On  large,  complex  projects,  get  similar  inputs  from  responsible 
task  leaders.  Include  requirements  specified  by  the  sponsor.  Table  XX- 1 
presents  a matrix-type  checklist  useful  for  this  purpose. 

2.  Rigorously  evaluate  the  need  for  each  data  item  on  the  shopping  list.  Who 
will  need  it?  When?  Why?  What  or  who  will  suffer  without  it'.’  How  much.’ 
Will  the  information  possibly  be  available  elsewhere  in  other  project  documen- 
tation. or  in  documents  already  in  the  government’s  possession?  In  other 
words,  determine  as  well  as  you  can  the  true  essentiality  of  each  data  item. 
Use  an  arbitrary  scale  of,  say,  five,  and  assign  a value  to  each  item. 

3.  Estimate  the  costs  of  engineering  anil  publication  efforts  on  each  data  item, 
weigh  against  the  essentiality  value  given  the  item:  and  obtain  a relative  cost- 
effectiveness  figure  for  the  item. 
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TABLi:  XX- 1.  DOCUMENTATION  TENTATIVE-REQUIREMENTS  PLANNING  MATRIX. 


Type  of  Document 


film 


4.  Make  initial  decisions  for  or  against  inclusion  of  the  items  and  establish 
an  initial  “minimum  essential”  data  requirements  list.  When  a contractor 

is  involved  and  the  total  contract  exceeds  SI  million,  contract  data  require- 
ments must  be  approved  by  a data  requirements  review  board  as  explained 
in  NELCINST  4000.1,  Data  and  Configuration  Management  Manual  (to  be 
issued). 

5.  Subject  the  initial  data  requirements  list  to  continuous  review  and  valida- 
tion in  the  light  of  project  developments. 

b.  When  ordering  data  from  contractors,  specify  use  of  the  contractor's  format 
if  it  is  suitable  and  the  data  item  is  to  be  used  solely  within  the  government. 
Impose  mil  spec  and  standard  content  and  format  requirements  only  for 
specifications,  drawings,  manuals,  etc,  that  must  meet  very  strict  needs  for 
procurement,  operation,  and  support.  Manuals  should  be  procured,  at  NOSC. 
through  Code  6400. 

7.  On  Rf-’Ps,  encourage  contractors  to  submit  alternative  data  packages  to  the 
data  package  represented  by  the  Contract  Data  Requirements  List  (DD  Form 
1423).  Negotiate  in  favor  of  the  least-expensive  package  submitted  which 
satisfies  the  government’s  needs  as  specified  in  the  statement  of  work. 

X.  If  contractors  are  to  provide  data  items,  determine  as  early  as  possible  when 
the  items  are  needed  for  use.  Order  them  under  the  appropriate  contract 
and  have  them  delivered  as  they  are  needed.  These  practices  will  help  to 
assure  that  the  government  obtains  data  at  the  lowest  possible  cost,  that  the 
delivered  data  are  usable  rather  than  in  need  of  revision,  and  that  govern- 
ment storage  and  handling  are  minimized. 

TYPICAL  DATA  ITEMS 

Table  XX-1  contains  a list  of  subjects  of  concern  to  project  managers  and  engi- 
neers. The  subjects  to  be  dealt  with  will  vary  with  the  project.  The  numbers  in  the  boxes 
of  the  table  correspond  to  the  numbers  in  the  KEY  column  of  table  XX-2  and  represent 
Data  Item  Descriptions  (DIDst  listed  in  the  table.  The  listed  DIDs  have  been  selecteil  from 
among  those  authorized  by  l)ol)  for  use  by  system  equipment  developers.  Altogether,  the 
DoD  Acquisition  Management  Systems  and  Data  Requirements  Control  I ist  (AMSDI  I 
DoD5000. 1 6L  of  January  1 1>77  and  the  NOS( ’ book  of  unique  DIDs  currently  contain  more 
than  1300  DIDs.  and  the  173  listed  in  table  XX-2  represent  only  the  more  significant  and 
useful  items  currently  available.  The  AMSDI.  should  be  consulted  for  the  most  current 
DIDs  and  revisions. 

Documents  covering  the  subjects  listed  in  table  XX- 1 can  be  grouped  into  three 
broad  categories  according  to  their  purpose. 

Those  describing  systems  and  equipment 

Those  used  to  control  design  changes 

I hose  required  to  support  design,  operation,  maintenance,  and  support 

The  primary  documents  in  each  category  are  discussed  briefly  in  the  following  para- 
graphs. Administrative  financial  documents  are  not  discussed;  particular  NOSC  require- 
ments in  this  area  are  covered  by  Ni  l CINST  5000.2.  primarily  in  chapters  VI  and  VII. 
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TABLE  XX-:.  REPRESENTATIVE  DATA  ITEM  descriptions  data. 


KEY 

DID 

TITLE 

SUBJECT 

1 

Dl-A-5239 

Management  Plan 

Administration.  Financial 

-) 

DI-A-1004 

Work  Breakdown  Structure 

3 

D1-A45  12 

Project  Status  Report 

4 

UDI-A-1 18 

Project  Progress  Report 

5 

Dl-A-2090 

Report.  Progress.  Letter  Type 

6 

DI-A-5024 

Preliminary  PERT/Time  Network  & Analysis 
Report 

*7 

DI-A-5025 

Periodic  PERT/Time  Network  & Analysis 
Report 

8 

D1-F-1208A 

Performance  and  Cost  Report 

9 

DI-A-4500 

Financial  Management  Report 

10 

Dl-F-4576 

Cost  Incurred  on  Contract  (DD  Form  1177) 

1 1 

Dl-F-6128 

Provisioning  Data  and  Repair  Parts  Funding 
Report 

12 

Dl-F-6126 

Report  of  Technical  Manual  Costs 

13 

DI-F-2014 

Report,  Composition  Cost 

14 

Dl-A-2007 

Chart,  Milestone 

15 

DI-A-5007A 

Document  Control  Data  RDT&E(DI)  Form 
1473) 

Historical 

16 

D1-S-I808A 

Design  Data  Summary  Sheet 

17 

DI-S-3582 

Subsystem/Engineering  Development  Report 

18 

DI-A-501 1 

Conference  Reports 

19 

DI-E4544 

Specification  (Program  Peculiar)  (Types  A.  B. 
and  C) 

Engineering  Design 

20 

DI-E4545 

Purchase  Description 

21 

D1-E4529 

Engineering  Drawings  (Cat  A),  Design  Evaluation  ' 
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DI-E4530 

Engineering  Drawings  (Cat  B).  Interface  Control 

23 

DI-E4531 

Engineering  Drawings  (Cat  C).  Service  Test 

Replaced  for  New  Design 

24 

DI-E4532B 

Engineering  Drawings  (Cat  E),  Procurement  of 
Identical  Items 

Engineering  Drawings 

25 

DI-E4533 

Engineering  Drawings  (Cat  F).  Procurement 
(Interchangeable  Items) 

DIE-7013  Level  1 
, 1)1-1-7014  Level  2 

26 

DI-E4534 

Engineering  Drawings  (Cat  C • ).  Installation 

Dl-1-7015  Level  3 

27 

DI-L4535 

Engineering  Drawings  (Cat  III.  Maintenance 

(reteience  Mil  S 1 D 1 00B  \ 

28 

Dl  l 4536 

1 ngineering  Drawings  (Cat  1).  Government  Manu- 
facture 

MIL-D-1000A) 

29 

DI-E4537 

Engineering  Drawings  (Cat  J ).  Interchangeabiliiv 
Control ) 

30 

DI-E-5056A 

Preparation  of  Logic  Diagrams 

XX4 


TABLE  XX-2.  (Continued) 


KEY 

DID 

TITLE 

31 

D1-E4538 

Index  List 

UD1-E- 
22708B  or 
DI-L-S074 

Nonstandard  Parts  Approval  Submissions 

Dl-V-2075 

Certification  of  Prior  Submission 

32 

DI-E4534 

Parts  List 

33 

DI-E4540 

Data  List 

34 

Dl-S-3618 

System  Engineering  Management  Plan 

35 

Dl-S-3564 

System/Cost  Effectiveness  Program  Plan 

36 

DI-S-3607 

Schematic  Block  Diagrams 

37 

DI-S-3604 

Functional  Flow  Diagrams 

38 

Dl-S-3606 

System/Design  Trade  Study  Reports 

34 

Dl-S-3614 

Technical  Performance  Measurement  Report 

40 

Dl-S-3540 

System/Subsystem  Summary  Report 

41 

DI-L-6138 

Integrated  Support  Plan 

42 

Dl-A-5345 

Design  to  Cost  Data 

43 

DI-A-5303 

Cost  Allocations  Report 

44 

DOD1  7041 .3,  Economic  Analysis  and  Program 
Evaluation  for  Resource  Management 

45 

LCC-1,  DoD  Life  Cycle  Costing  Procurement 
Guide  ( Interim) 

46 

LCC-2.  DoD  Casebook  Life  Cycle  Costing  in 
Equipment  Procurement 

47 

LCC-3.  DoD  Life  Cycle  Costing  Guide  for 
Systems  Acquisitions  ( Interim) 

48 

DI-R-2 1 13 

Plan,  Reliability  Program 

44 

DI-R-2 124 

Plan.  Reliability  Test 

50 

DI-R4808 

Reliability  Test,  Demonstration,  and  Evaluation 
Plan 

51 

Dl-R-3546 

Reliability  and  Maintainability  Analysis  for 
Exploratory/Advanced  Development  Model 

52 

DI-R-2 1 14 

Report.  Reliability  Allocation 

53 

DI-R-2 1 15 

Report,  Failure  Mode,  Effects,  and  Criticality 
Analysis 

54 

DI-R-2 1 1 7 

Report,  Reliability  Prediction 

55 

DI-R-21  18 

Report,  Reliability  Test  Result 

56 

DI-R-2 11 V 

Report.  Reliability  Status 

57 

DI-R-2 120 

Report,  Parts  Reliability 

58 

DI-S4855 

Maintainability  Program  Plan 

54 

1)1  R 3538 

Reliability  Maintainability  Demonstration 
Plan 

60 

Dl  R 5142 

Maintainability  Demonstration  Report 

SUBJECT 


System  Engineering 


Life  Cycle  Costs 


Reliability 


Maintamabilils 
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TABLE  XX-2.  (Continued)- 


KEY 

DID 

TITLE 

61 

DI-T-3708 

ETivironmental  Test  Plan/Proeedure 

62 

D1-F.-1 1 19 

Environmental  Criteria  Report 

63 

DI-R-21 16 

Report.  Environment.  Reliability 

64 

DI-H-2104 

Plan.  Human  Engineering  Program 

65 

D1-H4605 

Human  Factors  Engineering  Design  Approach 
Document 

66 

DI-H-2 105 

Plan.  Human  Engineering  Test  - 

67 

DI-H-4604 

Human  Factors  Engineering  Progress  Reports 

68 

DI-H-2 1 1 1 

Report,  Human  Engineering  Test 

69 

DI-H-4601 

Human  Factors  Engineering  Final  Report 

70 

Dl-A-5323 

Computer  Software  Project  Planning  Chart 

71 

UDI-E-141 

System  Operational  Specification 

72 

UDI-E-142 

System  Operational  Design 

73 

UDI-E-143 

Functional  Operational  Specification 

74 

UDI-E-144 

Interface  IX’sign  Specification 

75 

UDI-E-I45 

Program  Performance  Specification 

76 

UDI-E-146 

Function  Operational  Design 

77 

UDI-E-147 

Program  Design  Specification 

78 

UDI-E-148 

Program  Description  Document 

79 

UDI-E-149 

Data  Base  Design  Document 

80 

UDI-E-1 50 

Program  Package 

81 

Dl-R-2060 

Plan.  Design  and  Development.  Emission  Control 

82 

Dl-R-2061 

Plan.  Control.  Electromagnetic  Environment 

83 

DI-R-2064 

Plan.  Control.  Electromagnetic  Interference 

84 

DI-R-2056 

Plan.  Control,  Electromagnetic  Compatibility 

85 

DI-R-2058 

Plan.  Test.  Emission  Control 

86 

DI-R-2062 

Plan.  Test.  Electromagnetic  Environment 

87 

DI-R-2065 

Plan.  Test,  Electromagnetic  Interference'!  lectro 
magnetic  Compatibility 

88 

DI-R-2059 

Report.  Test.  Emission  Control 

89 

DI-R-2063 

Report.  Test.  Electromagnetic  Environment 

90 

1)1  R 2066 

Report.  Test.  Electromagnetic  Interference/ 
1 lectromagnetic  Compatibility 

9] 

DI-T4910 

Integrated  l est  Plan 

92 

DI-T4905 

Design  or  Engineering  Test  Plan 

93 

DI-T490 1 

first  Article  Inspection  Procedure 

94 

DI-T4903 

Production/ Acceptance  Inspection  Procedure 

95 

DI-T  2072 

Reports.  Test 

SCBJICT 

Environment 


Human  Engineering 


Software 


I Ml  I Ml  I MC 


Testing 
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TABLE  XX-2.  (Continued) 


KEY 

DID 

TITLE 

SUBJECT 

do 

DI-T4002 

First  Article  Inspection  Report 

d7 

DI-T-4004 

Production  Inspection  Reports 

ds 

DI-E4540A 

End  Item/Equipment  Identification 

Nomenclature  Assig n men t 

dSA 

DI-E4542 

Identification  Plates 

dd 

Dl-E-5 1 85 

Plan  for  Assignment  of  Ref  Designations 

100 

DI-E-0 1 26A 

Request  for  Nomenclature 

101 

DI-S-6170A 

Facilities  Requirements  Plan 

Facilities 

102 

DI-V4058B 

Item  Identification 

Support  Equipment.  Items 

100 

DI-V-2074 

Support  Equipment  List 

104 

DI-V-7006 

Interim  Support  Items  List 

105 

DI-V-2080 

Interim  Repair  Parts  List 

Provisioning 

106 

DI-V-7002 

Provisioning  Parts  List  and  DI-V-7000  (Short  Form) 

107 

DI-V-7004 

Long  Lead  Time  Items  List 

10S 

Dl-V-7005 

Repairable  Items  List 

100 

Dl-V-7007 

Tools  and  Test  Equipment  List 

1 10 

DI-V-7008 

Common  and  Bulk  Items  List 

1 1 1 

DI-V-7000 

Design  Change  Notices 

1 1 2 A 

D1-V4052A 

Provisioning  Technical  Documentation  (Army) 

1 1 2B 

DI-V4O50A 

(Navy) 

1 1 2C 

DI-V4054A 

(Air  Force) 

1 1 21) 

DI-V-7000 

(Supplementary) 

1 1 2E 

D1-V-7016A 

(DSA) 

1 1 2F 

DI-V-700 1 

(Commercial  Items) 

1 10 

Dl-1 1-200 1 

Training  Course  Plan 

Personnel  and  Training 

1 14 

Dill-6101 

Training  and  Training  Equipment  Plan 

115 

Dl  11-2106 

Report.  Personnel  Planning  Information 

116 

1)1  -11-2 100 

Report.  Task  Analysis/Task  Description 

117 

DI-H-2108 

Report . Human  Engineering  Maintenance 
Accessibility  Design 

1 IS 

Dl-1 1-2028 

Instructor’s  Guide 

1 10 

Dill-2020 

Learner’s  Guide 

1 20 

DI-L-6147 

Preservation  and  Packaging 

Pteservation  and  Packaging 

1 21 

DI-L402O 

Packing.  Marking,  and  Shipping  Data  Sheets 

Packing  and  Marking 

1 22 

DI-E-0 107 

Parts  Control  and  Standardization  Plan 

Standatdizatton 

120 

1)1-1  -1  1 1 6A 

Standardization  Component  Selection  and 
Control 

I24A 

1)1-1  1 1 1 7 A 

Standardization  Report  ol  Common  Ileitis 

I24H 

Dl  l 01  2411 

Standardization  Parts  Selection  Ptogram 
(Mil  SID  S»|  , 
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TABLE  XX-2.  (Continued) 


KKY 

Dll) 

TITLE 

SUBJECT 

I24C  D1-E-3137A 
124D  DI-E-3 1 25A 
1 24F  DI-E-3 134 

Parts  Control  and  Standardization  Plan 
Nonstandard  Parts  t'or  New  Design 
Parts  Control  Summary 

125 

Dl-R-3527 

System  Security  Plan 

Security 

126 

Dl-S-5  137  A 

Security  Failure  Analysis  Report 

127 

DI-H-1320A 

System  Safety  Plan 

Safety 

128 

Dl-H-1 322A 

Safety  Statement 

124 

DI-L-6143 

Logistics  Support  Plan  for  Preoperational 
Support 

Operation  and  Maintenance 

130 

DI-S-6168 

Maintenance  Engineering  Analysis  Program  Plan 

131 

Dl-S-6171 

Maintenance  Engineering  Analysis  Data 

132 

Dl-L-2084 

Plan,  Level  of  Repair  Program 

133 

DI-L-2085 

Report,  Level  of  Repair  Analysis 

134 

DI-L-2082 

Report,  Level  of  Repair  Summary 

135 

DI-L-2101 

Standard.  Technical  and  Maintenance  Overhaul 
and  Repair 

136 

Dl-M-6154 

Technical  Manual  Plan 

136 

Dl-M-6157 

Technical  Publications  for  Advanced  Development 
Program 

' 

137 

UDI-M-1 15 

Manual,  Technical,  Systems  and  Equipment 

138 

UDI-M-1 16 

Manuscript,  Technical  Publications 

134 

UDI-M-1 17 

Technical  Manuals,  Camera  Ready  Copies  of 

140 

Dl-M-6154 

Validation  Record  (Technical  Manuals) 

141 

DI-M4712 

Operating  and  Maintenance  Instructions 

142 

UDI-M-102 

Commercial  Technical  Manuals 

143 

IJDI-M-104 

Operator's  Ouide 

144 

UDI-M-1 10 

Computer  Program  Operator’s  Manual 

145 

UDI-M-1 1 1 

System  Operator's  Manual  (Computers) 

• 

146 

UDI-M-1 14 

Computer  Program  Operator's  Guide 

147 

DI-R-4802 

Quality  Assurance  Program  Plan  Requirements 

Quality  Control 

148 

DI-R4803 

Inspection  System  Program  Plan 

144 

DI-R4804 

Calibration  Data  & Procedures 

150 

DI-R4807 

Design  Review  Schedule 

151 

DI-R48 10 

Design  Review  Report 

152 

DI-R4804 

End  Item  Final  Inspection  Record 

153 

DIP  1602 

Value  Engineering  Plan 

Value  Engineering 

154 

DI-A-3004 

Value  Engineering  Progress  Report 

155 

Dll’  1600 

Value  Engineering  Data  Report 

156 

Dl-I  2035 

Plan,  Configuration  Management 

Configurat ion  Control 
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TABLE  XX-2.  (Continued) 


KEY 

DID 

TITLE 

SUBJECT 

157 

DI-E-2037 

Engineering  Change  Proposals  and  Requests 
for  Deviations  and  Waivers  (Long  Form) 

158 

Dl-E-2038 

Engineering  Change  Proposals  and  Requests 
for  Deviations  and  Waivers  (Short  Form) 

159 

DI-E-2039 

Reports,  Configuration  Status  Accounting 

160 

Dl-A-3029 

Agenda  Design  Reviews,  Configuration  Audits 
and  Demonstrations 

161 

Dl-E-3 1 18 

Minutes  of  Formal  Reviews,  Inspections  and  Audits 

162 

Dl-A-3022 

Contract  Data  Management  Plan 

Data  Management 

163 

Dl-A-3027 

Data  Accession  List/Internal  Data 

164 

DI-E-5249A 

Engineering  Data  Abstract  and  Document 
Record 

165 

DI-P-16 12 

Production  Plan 

Manufacturing,  Production 

166 

Dl-P-3455 

Production  Analysis  Report 

167 

DI-P-4752 

Production  Progress  Reporting 

168 

DI-P-3472A 

Procurement  Data  Package  and  Lists 

Procurement , Reprocurement 

169 

DI-A-6101 

Contractor  Engineering  and  Technical 
Services  Plan 

170 

DI-A-3003 

Performance  Incentive  Analysis  Report 

171 

Dl-E-3 144  A 

Engineering  Data  for  Research,  and  Prototype 
Hardware 

Research  & Development 

172 

DI-E-61 16 

Data,  Engineering,  Reports,  and  Lists  for  R&D 
Work  Performed  at  Govt  expense 

173 

DI-A-3002 

R&D  Status  Report 

SYSTEM/EQUIPMENT  DESCRIPTION 

SPECIFICATIONS 

Specifications  provide  narrative  details  of  performance  and  physical  character- 
istics and  their  quality  assurance  requirements  for  systems,  equipments,  assemblies,  sub- 
assemblies, components,  etc.  Development-type  specifications  are  prepared  to  establish 
initial  baseline  configurations  for  developing  prototype  models.  Product-type  specifica- 
tions reflect  the  finally  developed  configurations  and  are  used  to  procure  operational 
models.  Specifications  are  prepared  to  commercial/industrial  or  military  standards. 
Program-peculiar  military  specifications  are  required  by  MIL-S-8.3490.  which  covers  the 
requirements  for  Forms  la.  lb.  2.  and  3 and  Types  A.  B.  C.  D.  and  I specifications. 

The  detail  requirements  for  Form  la  specifications  of  all  types  are  presented  by  MIL-STD- 
4lH).  while  MIL-STD-hpl  provides  overall  requirements  for  military  specifications  in  general. 
Ml L-STl )-'>b 2 provides  requirements  for  military  handbooks. 

An  engineer,  whether  he  is  a circuit  designer  or  has  the  total  responsibility  for  the 
overall  system  design,  should  be  familiar  with  MIL-S  IT)-4lH).  since  it  provides  the  guidance 


for  writing  specifications  which  will  be  followed  throughout  the  various  stages  ol  military 
system  development.  A specification  which  is  not  comprehensive  (ie.  does  not  cover  such 
requirements  as  reliability,  human  engineering,  support  documentation,  maintainability,  or 
logistics)  will  practically  guarantee  higher  support  costs  during  a system's  scheduled  lifetime 
and  may  also  shorten  the  duration  of  that  lifetime. 


ENGINE E RING.  DRAW  1NGS 


Engineering  drawings  are  the  designer's  primary  means  ot  communication.  They 
may  be  prepared  in  accordance  with  commercial/industrial  or  military  standards.  T he 
Department  of  Defense  recognizes  10  general  categories  ot  drawings  based  on  intended 
use  (M1L-D-1000).  These  categories  may  contain,  variously,  several  of  46  different  types 
of  drawings  (MIL-STD-100).  Military-required  drawings  may  be  prepared  to  Form  1,  2. 
or  3 standards  as  specified  by  MIL-D-1000. 


TEST  PROCEDURES  AND  REPORTS 

Procedures  and  reports  are  required  for  tests  of  specified  pertormance  and  phy- 
sical characteristics  for  developmental  and  production  models.  They  are  prepared  by 
the  development  activity  and  manufacturer  in  accordance  with  their  own  or  the  buyer’s 
requirements.  Various  requirements  documents  of  the  several  military  services  and 
agencies  address  the  content  and  format  of  test  procedures  and  reports  and  may  be  im- 
posed on  the  preparer. 


ENGINEERING  REPORTS 

During  equipment  development,  reports  are  often  required  by  the  purchaser  on 
such  engineering  aspects  as  reliability  and  maintainability  predictions  and  demonstration 
human  engineering,  and  satety.  The  specialty  engineers  involved  prepare  such  reports 
to  their  own  in-house  requirements  or  may,  in  the  case  of  items  tor  the  military  , have 
to  comply  with  applicable  military  specifications  and  standards. 


ENGINEER’S  NOTEBOOK 

An  engineer’s  notebook  should  be  established  the  moment  a new  design  is  be- 
gun. Instructions,  dates,  findings,  decisions,  reasons,  conclusions,  etc.  should  be  included 
A notebook's  value  varies  with  the  circumstances  and  the  extent  of  its  contents.  Poten- 
tial benefits  of  a notebook  include  patent  rights,  applications  to  future  projects,  detense 
of  actions  taken,  information  for  new  personnel,  and  general  future  reference. 


DESIGN  CHANGE  CONTROL 

Equipment  configuration  management  is  a normal  activity  ot  most  companies 
and  government  developers.  Effective  configuration-change  control  employs  a system  ot 
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recorded  change  data  subject  to  review  and  approval.  The  design  engineer  provides  the 
technical  inputs  for  this  documentation. 


ENGINEERING  CHANGE  PROPOSALS  ( l C Ps) 

Configuration  changes  are  formally  initiated  by  a change-proposal  form/ 
document  which  must  be  reviewed  and  approved  before  the  subject  change(s)  can  be 
incorporated  in  the  design  or  equipment.  The  designer  himself  may  initiate  the  pro- 
posal or  may  cooperate  with  another  engineer  who  has  the  original  idea  for  a change. 

In  either  case,  the  designer  must  describe  how  the  proposed  change  affects  his  area  of 
design  responsibility  by  completing  the  change  proposal  form/document  and  including 
appropriate  drawing  and  specification  ( see  SPLCIFICATION  CHANGE  NOTICES)  re- 
visions for  reference  as  well  as  required  costs  and  schedule  revision  information.  The 
military  services  must  follow,  and  cite  to  contractors,  the  requirements  of  MIL-STD-480 
or  MIL-STD-I8I.  which  employ  DD  Form  1692  or  1693.  respectively,  as  the  engineering 
change  proposal  document.  Variations  of  these  forms  may  be  used  by  industry  and 
DoD  for  in-house-only  applications.  The  completed  form/document  is  reviewed  by  a con- 
figuration control  board  which  may  immediately  decide  for  or  against  the  proposal  or 
return  it  to  the  originator(s)  with  a request  for  additional  information.  A signed,  ap- 
proved ECP  constitutes  authorization  to  proceed  with  the  change,  a process  that  nor- 
mally includes  the  preparation  of  engineering  change  orders  and  specification  change 
notices  (see  following). 


ENGINEERING  CHANGE  ORDERS 

Properly  authorized  engineering  change  orders  provide  the  technical  and  pro- 
cedural information  necessary  to  fully  implement  the  authorized  change  to  the  design/ 
equipment  and  to  the  engineering  documentation  that  baselines  the  configuration  ot  the 
design/equipment.  An  engineering  change  order  may  be  called  a “Notice  of  Revision." 
“Engineering  Change  Notice,"  “Revision  Directive,"  or  other  similar  terms  according 
to  local  practice.  The  designer  will  be  required  to  contribute  technical  information 
within  his  purview  to  this  document. 


SPECIFICATION  CHANGE  NOTICES  (SCNs) 

Whenever  a change  to  a design  or  equipment  is  authorized,  the  pertinent  speci- 
fication and  engineering  drawings  must  be  changed  accordingly.  Instructions  for  docu- 
ment revision  should  be  contained  in  engineering  change  orders.  An  approved  SCN 
sheet  should  be  included  with  these  instructions  along  with  the  detailed  information 
needed  to  produce  revised  copy  for  the  specification.  The  SCN  sheet  and  revised  pages 
prepared  for  the  specification  eventually  are  attached  to  the  latest  approved  version  of 
the  specification,  thus  updating  the  specification  to  agree  with  the  updated  configura- 
tion of  its  item.  Military  program-peculiar  specifications  must  be  changed  only  by  using 
DD  Form  1696  as  prescribed  by  MIL-STD-490.  Changes  to  specifications  not  under 
military  control  may  be  controlled  by  other  SCN  practices  peculiar  to  the  individual 
developer  or  manufacturer.  In  any  event,  the  designer  may  be  called  upon  to  prepare 
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or  review  revised  portions  of  specifications  as  required  by  SC'Ns  (which  themselves  are 
usually  prepared  by  configuration  management  personnel). 

SUPPOR 1 DOCUMENTATION 

While  the  engineering  data  and  design  change  documentation  delineated  above 
are  primarily  applicable  to  the  design  and  development  cycles,  the  design  engineer  may 
be  called  upon  to  provide  engineering  data  and  information  required  lor  the  preparation 
of  such  support  documentation  as  technical  manuals  for  operation  and  maintenance, 
maintenance  standards  books,  computer  programming  documentation,  operating  instruc- 
tion charts,  and  field  change  documents. 

TECHNICAL  MANUALS 

The  design  engineer  generally  is  not  required  to  prepare  such  documentation  him- 
self. as  technical  manual  writing  is  a specialty  occupation,  but  he  may  be  required  to 
• develop  the  source  data  for  the  manuals  and  is  in  the  best  position  to  review  and  approve 
manuscripts  for  completeness,  accuracy,  and  adequacy  ol  purpose.  Commercial  manuals 
purchased  with  commercial  off-the-shelf  items  should  be  required  to  meet  the  minimum 
standards  for  such  documentation,  as  delineated  it'  MIL-M-7298.  Manuals  for  systems 
and  equipment  developed  by  or  for  the  military  services  must  be  prepared  in  accordance 
with  applicable  military  specifications  and  standards.  Each  service  has  its  own  require- 
ments reflected  by  its  own  “limited”  specifications.  The  military  specification  most  com- 
monly used  for  Navy  electronic  equipment  manuals  is  MIL-M-15071.  Technical  manuals 
should  be  procured  at  NOSC,  through  NOSC  Code  6400  (Technical  Information  Division). 


MAINTENANCE  STANDARDS  BOOKS 

These  documents  provide  equipment  maintenance  instructions  to  Navy  person- 
nel in  the  field  when  the  Programmed  Maintenance  System  is  not  in  effect.  While  these 
documents  can  be  prepared  mainly  from  data  already  in  the  associated  technical  manuals, 
engineers  may  be  required  to  provide  additional  information  to  the  technical  writers 
Maintenance  standards  books  are  prepared  in  accordance  with  MIL-B-21  741 

OPERATING  INSTRUCTION  CHARTS 

Usually  prepared  by  technical  writers  from  information  available  in  technical  man- 
uals, these  charts  may  on  occasion  require  some  input  from  an  engineer 
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If  equipment  already  in  the  field  is  to  be  modified,  the  personnel  responsible  lor 
making  such  modifications  require  clear,  complete  instructions  as  to  the  purpose  and 
method  of  modification,  checkout  after  modification,  and  other  factors.  In  addition. 
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existing  technical  manuals  and  other  support  documentation  in  the  field  will  require  changes 
that  reflect  the  modification.  Engineers,  whether  or  not  they  were  involved  in  the  initial 
design  of  the  subject  equipment,  may  have  to  provide  to  the  technical  writer  all  the  informa- 
tion required  by  the  specification  to  which  the  field  change  and  technical  manual  change 
documentation  is  prepared.  The  specification  for  field  changes  is  M1L-F-17655. 


COMPUTER  PROGRAMMING  DOCUMENTATION 

When  the  design  of  a computer  or  any  associated  system  is  accomplished,  cer- 
tain types  of  “user”  documents  are  required  to  ensure  that  operating  and  maintenance 
personnel  have  the  required  information  for  the  performance  of  their  mission.  (These 
“user”  documents  are  in  addition  to  the  programming  specifications  that  must  be  pre- 
pared during  the  design  cycle.)  While  responsibility  for  providing  the  data  and  infor- 
mation for  user  computer  program  documentation  usually  falls  primarily  on  the  pro- 
grammer. the  character  of  the  user  documents  is  such  that  both  the  programmer  and 
the  hardware  design  engineer  have  to  contribute  to  the  inputs  furnished  the  technical 
writer.  They  should  also  review  such  documents  for  completeness,  accuracy,  and 
adequacy.  Depending  upon  the  type  and  application  of  the  computer-associated  equip- 
ment in  question,  there  are  several  specifications  or  instructions  covering  the  require- 
ments for  content  and  format  of  the  various  documents  required.  These  include  SEC- 
NAVINST  5233.1  A.  Weapons  System  Specification  WS-8506.  and  SECNAV1NST 
3560.1. 


PARTS  PROVISIONING  DOCUMENTATION 

The  engineer  should  be  aware  of  the  requirements  for  provisioning.  Provisioning 
documentation  is  a direct  result  of  maintainability  tasks  which  determine  the  type  and 
level  of  maintenance  actions  required  for  a design.  The  engineer’s  direct  input  to  main- 
tainability tasks,  therefore,  indirectly  affects  provisioning  actions  and  associated  costs. 
Provisioning  documentation  is  generally  prepared  in  accordance  with  M1L-STD-1561  by 
a technical  writer  who  specializes  in  this  field. 


OTHER  SUPPORT  DOCUMENTATION 

The  engineer  must  sometimes  provide  data,  information,  and  guidance  for  the 
preparation  of  documents  other  than  those  described  in  previous  subsections.  Reference 
standards  books,  operators’  handbooks,  maintenance  manuals,  and  training  guides  are 
examples  of  these,  along  with  other  specialized  documents  required  for  training,  instal- 
lation, operation,  and  maintenance  in  the  field.  Support  documentation  is  prepared 
mainly  by  technical  writing  specialists:  however,  the  engineer  is  expected  to  provide  per- 
tinent information,  and  in  some  cases  to  contribute  written  material. 
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PROJECT  MANAGEMENT  SUPPORT  DOCUMENTATION 

Plans  in  considerable  variety  are  normally  required  to  support  the  project  man- 
agement function.  These  plans  should  be  kept  to  a minimum;  however,  they  are  some- 
times important  checks  on  contractor  procedures 

Configuration  Management  Plan 
Configuration  Status  Accounting  Reports 
ILS  Plan 

Design-to-Cost  Data 

Reliability  Program  Plan 

Maintainability  Program  Plan 

Environmental  Test  Plan/Procedures 

Human  Engineering  Program  Plan 

Maintenance  Engineering  Analysis  Program  Plan 

LOR  Program  Plan 

QA  Program  Plan 

Inspection  System  Program  Plan 

Production  Plan 

ACCESSION  LIST 

The  use  of  an  accession  list  is  highly  recommended.  An  accession  list  is  a per- 
iodic report  of  all  the  documents  which  a contractor  produces  for  internal  use;  the  gov- 
ernment obtains,  through  the  accession  list,  access  to  this  information  for 
only  the  cost  of  reproduction.  The  data  required  by  the  government  in  the  contract  are 
limited  to  only  known  essential  data  items,  thus  reducing  costs  by  not  asking  for  data 
which  may  or  may  not  be  needed.  The  Statement  of  Work  must  contain  the  following 
task:  “The  contractor  shall  prepare  and  update  (monthly)  a list  of  all  reports,  records, 
memoranda,  plans,  procedures,  and  other  data  items  generated  in  support  of  this  contract. 
Upon  request  from  the  Government,  the  contractor  shall  provide  copies  of  specific  data 
items.  The  Government  shall  reimburse  the  contractor  for  reproduction  costs  incurred  in 
this  task."  The  CDRL  must  reference  the  task  in  the  SOW.  The  CDRL  item  for  an 
Accession  List  Data/Internal  Data  can  cite  DIDs  DI-A-3027  or  UDI-A-26486. 


XX-14 


XXI.  RISK  MANAGEMENT 

GENERAL  SUMMARY  OF  RISK  CONSIDERATIONS  . . . page  XXI-1 
RISK  IDENTIFICATION  . . . XX\A 

RISK  CHARACTERIZATION  AND  ASSESSMENT  . . . XXI-5 
HANDLING  RISK  . . . XX1-6 

BETTER  ESTIMATES  USING  RISK  ASSESSMENTS  . . . XXI-8 


A 


XXL  RISK  MANAGEMENT 


XXI.  RISK  MANAGEMENT 


The  project  manager  is  charged  with  the  responsibility  to  manage  the  tasks,  funds, 
schedules,  and  risks1.  Risks  are  encountered  by  everybody,  everyday,  in  everything  they 
do,  from  simply  getting  out  of  bed  to  taking  a trip  to  the  moon.  Most  risks  are  so 
small  or  inconsequential  that  most  everyone  ignores  them;  even  the  sometimes  painful 
consequences  are  shaken  off  as  bad  luck.  In  a classic  example,  one  seldom  considers  the 
risk  of  pain  when  driving  a nail  to  hang  a picture,  and  a throbbing  finger  easily  obscures 
the  concept  that  a risk  was  taken.  A complex  technical  project  deserves  better  attention 
to  risk  than  driving  a nail,  and  recognizing  that  any  project  entails  many  risks  highlights 
the  need  for  risk  management. 

Risk  management  has  been  developed  into  a highly  effective  tool  by  most  success- 
ful project  managers,  although  its  application  is  often  intuitive.  However,  the  insurance 
industry  has  consciously  developed  risk  management  into  a rigorous  science.  The  decisions 
facing  a technical  project  manager  will  seldom  yield  to  the  statistical  analysis  of  an  actuary 
but  the  qualitative  techniques  which  have  evolved  are  very  useful.  The  sections  which 
follow  discuss  risk  management  as  it  can  be  applied  to  technical  projects.  If  this  guidance 
at  least  stimulates  conscious  consideration  of  risk  with  the  other  project  factors,  it  will 
have  served  its  purpose. 


GENERAL  SUMMARY  OF  RISK  CONSIDERATIONS 

Risk  is  a probability  of  failure  to  achieve  a well  defined  goal.  If  success  is  the 
unqualified  achievement  of  the  goal,  the  probability  of  success  is  one  minus  the  risk, 
since  all  probabilities  are  measured  on  a scale  from  zero  (no  chance  of  occurrence)  to  one 
(certainty  of  occurrence).  Success  and  failure  hinge  on  the  definition  of  the  goal;  the  well 
defined  goal  consists  of  the  following  characteristics: 

A complete  description  of  that  to  be  attained  including  tolerances  on  each  descrip- 
tive parameter 

The  resources  to  be  spent  in  attainment 
The  time  in  which  the  goal  is  to  be  reached 

Any  constraints  which  may  exist  on  the  actions  that  may  be  taken  to  achieve  the 
goal 

Obviously,  there  are  many  examples  of  partial  success  which  were  termed  either  satisfac- 
tory or  unsatisfactory:  this  directly  implies  two  underlying  risk  principles: 

1.  There  are  a number  of  elements  which,  taken  together,  describe  the  goal; 
there  is  a risk  associated  with  each  element. 

2.  There  are  consequences  attached  to  the  failure  to  attain  each  element  which 
may  vary  from  insignificant  to  catastrophic. 

These  seemingly  trivial  definitions  and  principles  form  the  foundations  of  risk  management 
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Since  risk  and  the  probability  of  success  sum  to  one,  it  is  c.isy  to  jump  to  the  con- 
clusion that  the  objective  of  risk  management  is  to  minimize  risk,  thereby  maximizing 
probability  of  success.  11  all  risk  could  be  reduced  to  zero,  success  would  he  certain. 
However,  it  would  seem  that  both  omniscience  and  omnipotence  would  be  necessary  to 
bring  this  condition  about.  More  practically,  some  finite  risk  must  always  he  assumed  to 
exist.  If  success  is  not  certain,  the  possible  consequences  of  failure  must  be  considered. 

This  adds  a new  dimension  to  risk  management  that  of  maximizing  the  probability  of 
an  acceptable  outcome;  this  may  even  mean  a lower  probability  of  success  per  se. 

This  leads  to  the  problem  of  defining  what  constitutes  an  acceptable  and  an  un- 
acceptable consequence.  For  purposes  of  this  discussion,  the  categories  will  he  defined  as 
follows: 

Critical  A failure  which  is  of  itself  unacceptable,  ie,  catastrophic 

Potentially  critical  A failure  which  is  acceptable  but  which  is  unacceptable 

in  combination  with  other  failures 

Probably  noneritical  A failure  which  is  acceptable  and  must  occur  in  com- 

bination with  several  other  failures  to  lead  to  an 
unacceptable  result 

Noneritical  A totally  inconsequential  failure 

Another  practical  problem  concerns  the  fact  that  everyday  goals  are  simply  not 
well  defined.  Indeed,  some  aspects  may  defy  practical  determination.  In  order  to  de\  op 
eriteria  for  success,  to  identify  risks,  and  to  determine  the  consequences  of  failure,  it  is 

usually  necessary  to  estimate  aspects  of  the  missing  pieces.  The  estimation  process  at  its 

best  relies  on  educated  guesses  and  at  its  worst  on  the  other  kind.  In  each  case,  the  prob- 
ability of  estimating  error  and  its  consequences  must  be  considered  in  the  risk  assessment. 

The  four  characteristics  formulating  the  well  defined  goal  are  closely  interrelated, 
thus,  the  associated  risk  elements  are  interrelated.  Such  functional  relationships  will  ob- 
viously affect  the  tactics  to  be  employed  in  managing  the  risks.  These  relationships  may 
he  categorized  in  the  following  manner: 

Independent  Not  affected  by  other  risk  elements 

Dependent  Totally  determined  by  the  outcome  of  other  elements 

Interdependent  Functionally  related  to  other  elements  such  that  each  alteets 
the  others 

Hie  nature  of  the  goal  determines  the  exact  relationships  as  well  as  the  consequences 
associated  with  its  risks.  Nevertheless,  the  cour-vs  T action  for  dealing  with  the  risks 
must  clearly  tackle  independent  elements  directly,  attack  interdependent  elements  as  a 
group,  and  approach  dependent  elements  indirectly  by  manipulating  the  elements  on  which 
they  are  dependent. 

If  each  of  the  four  goal  characteristics  is  a single  element,  it  is  relatively  simple  to 
analyze  the  goal,  to  assess  risks,  anil  to  assign  consequences  to  those  risks,  therefore,  the 
risks  should  he  fairly  simple  to  manage.  Unfortunately,  this  is  not  necessarily  the  case. 

1 ach  goal  element  is  toleranced:  ie,  a range  of  acceptability  is  defined  about  the  goal.  II 
the  tolerance  is  tight,  it  will  be  more  difficult  to  stay  within  the  range  than  if  the  tolerance 
is  loose.  When  the  elements  are  interrelated  so  that  one  must  be  traded  off  against  another, 
tight  tolerances  make  this  task  harder  It  the  tolerance  is  sufficiently  loose,  tradeoffs  might 
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bo  in  ado  such  that  the  probablo  oonsoquence  of  the  associated  risk  is  affected.  This  will 
depend  on  the  nature  of  the  risk,  its  relationships  with  other  risks,  and  the  range  of 
acceptability.  Because  the  manipulating  of  one  risk  to  change  another  appears  intuitively 
to  be  a potent  tool,  it  seems  wise  to  identify  those  risks  which  will  and  will  not  lend 
themselves  to  alteration  by  this  method,  accordingly,  the  risks  should  be  classed  as  follows: 

Rigid  Those  elements  for  which  the  risk  consequences  cannot  be  altered 

regardless  of  the  degree  of  risk  assessed  or  of  relationships  with 
other  elements 

Flexible  Those  elements  for  which  the  risk  consequences  can  be  altered 

A risk  which  is  critical  (or  potentially  critical)  but  flexible  is  obviously  a good  management 
target.  Conversely,  a rigid,  noncritical  risk  makes  an  excellent  tradeoff  against  related 
targets. 

The  more  complex  the  goal  is,  the  more  important  it  is  to  be  able  to  categorize 
goal  elements.  Within  each  goal  characteristic,  many  subdivisions  arc  normally  possible 
before  the  goal  is  partitioned  all  the  way  down  to  the  individual  element.  While  the  sub- 
divisions may  vary  with  each  field  of  endeavor,  the  keys  to  discovering  the  functional 
relationships  between  the  elements  will  probably  be  disclosed  by  studying  the  rules  of 
partitioning.  Nonarbitrary  rules  of  partitioning  must  be  invertible:  otherwise,  there  could 
be  no  was  to  synthesize  the  whole  from  the  parts.  The  inverted  rules  (rules  of  integration) 
will  describe  the  desired  relationships  by  necessity.  Since  these  relationships  are  required 
knowledge  for  the  risk  manager,  the  implication  is  that  the  risk  manager  must  be  con- 
versant with  that  field  of  endeavor.  Very  complex  goals  spanning  several  fields  will  require 
a multidisciplinary  background  for  the  management  team,  embodied  in  the  management 
leader. 

The  preceding  discussions  have  assumed  that  risk  management  can  be  reduced  to 
practice.  While  this  may  have  been  a brash  assumption  for  the  general  case,  it  is  hearten- 
ing to  note  that  an  entire  industry  is  built  on  risk  management  the  insurance  industry, 
(liven  the  apparent  success  in  that  field,  it  seems  reasonable  to  assume  that  somewhat 
effective  techniques  must  lie  available  to  manage  risks  in  a cognitive  manner  in  general. 

In  reviewing  the  general  discussions  above,  there  would  seem  to  be  at  least  eight  categories 
of  technique: 

Defining  the  goal 
Identifying  risks 
Assessing  consequences 
Assessing  degree  of  risk 
Manipulating  consequences 
Manipulating  degree  of  risk 
Assessing  results 

Developing  a best  plan  of  attack  to  achieve  a goal 

Both  qualitative  and  quantitative  methods  are  needed  in  order  to  adequately  address  the 
life  cycle  of  attaining  a goal  from  concept  to  success.  Such  results  should  describe  practi- 
cal guidelines  of  great  utility  to  the  technical  project  manager 
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RISK  IDENTIFICATION 


The  first  step  in  risk  management  must  be  the  identification  of  t he  risks  to  be 
managed.  Indeed,  once  a risk  is  identified,  its  management  often  becomes  obvious.  There 
is  no  cookbook  method  or  formula  to  identifying  risks:  at  any  given  time,  it  is  nearly 
impossible  to  identify  all  the  risks  for  a project.  However,  the  following  procedure  may 
be  easily  adapted  to  fit  the  peculiarities  of  the  project  and  aid  in  risk  identification. 

For  identification  purposes,  risks  may  be  considered  as  either  project  oriented  or 
technology  oriented.  Project-oriented  risks  are  those  which  are  associated  with  the  goals 
of  the  specific  project,  such  as  cost  targets,  schedules,  and  resource  constraints.  Tech- 
nology-oriented risks  are  those  which  would  be  essentially  the  same  regardless  of  the  pro- 
ject goals;  such  risks  are  usually  associated  with  achieving  a specified  level  of  performance. 
There  are  also  political  risks  which  take  on  the  character  of  either  project-  or  technology- 
oriented  risks.  Opposition  to  project  budget  appropriations  will  have  a project-oriented 
character;  for  instance,  opposition  to  the  operational  requirement  or  technology  animosity 
(as  against  nuclear  power)  may  be  regarded  as  technically  oriented. 

The  identification  of  project-oriented  risks  is  directly  facilitated  by  project  planning. 
The  better  a project  is  organized,  the  easier  and  more  certain  the  risk  identification  process 
can  be.  By  utilizing  the  work  breakdown  structure  (WBS)  organization  described  in 
chapter  111.  Program  Planning,  a risk  can  be  envisioned  for  the  performance,  cost,  schedule, 
and  availability  of  resources  for  each  individual  task  element.  In  practice,  the  risk  associ- 
ated with  many  elements  will  be  so  small  as  to  be  neglected.  However  before  throwing 
out  a risk,  consider  its  impact  on  the  next  higher  level  in  the  WBS:  if  the  impact  is  incon- 
sequential, the  risk  may  be  neglected.  Generally,  it  will  not  be  practical  to  consider  the 
tasks  below  level  4 in  the  WBS.  but  even  the  smallest  project  will  usually  be  concerned 
with  tasks  at  the  third  level.  It  is  handy  to  reproduce  the  project  WBS  and  to  circle  the 
elements  which  present  the  greatest  risk.  On  a complex  project,  a color  code  will  be  useful 
to  identify  each  risk  as  high,  medium,  or  low  and  the  risk  consequences  as  critical  semi- 
critical,  or  noncritieal.  Assume  that  opposition  will  develop  against  the  required  resources 
for  a task  to  determine  whether  political  risks  exist.  Usually,  such  risks  will  impact  on 
funding,  but  organizational  relationships  may  also  present  risks  in  accessing  certain  facili- 
ties or  personnel  talent. 

The  identification  of  technology-oriented  risks  is  controlled  by  the  knowledge  of 
that  technology  available  to  the  project.  The  more  information  that  is  assimilated  by  the 
project,  the  easier  it  is  to  identify  and  manage  the  technical  risks.  The  information  may 
come  from  several  sources: 

Knowledgeable  people 

Research 

Past  experience 

In  general,  the  gathering  of  information  requires  both  time  and  money.  If  a constant 
effort  is  expended  to  gather  information,  the  information  base  will  tend  to  grow  as  a 
logarithmic  function  of  time.  For  these  two  masons,  the  information  gathering  effort  is 
most  effective  at  the  start  of  a project.  To  limit  the  scope  of  effort  needed,  it  is  impor- 
tant to  utilize  talented  personnel  who  have  experience  in  the  technology  in  the  project 
planning  phases;  they  will  know  much  of  the  required  information  and  will  also  know  to 
a large  extent,  what  information  is  unknown.  Thus,  it  will  be  possible  to  organize  research 
efforts  to  gain  that  information  which  is  needed  most.  There  will  always  be  unanswered 
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questions  and  unknowns  to  which  the  project  will  be  naive,  so  some  risk  must  always  be 
incurred.  Chapter  IV.  Conceptual  Phase,  contains  guidelines  for  gathering  information  and 
for  organizing  the  technical  effort  in  accordance  with  the  perceived  risk.  The  fixed  politi- 
cal risks  must  be  discovered  by  knowing  the  project’s  “chain  of  command”  and  the  atti- 
tudes of  personnel  therein  either  directly  or  indirectly  and  also  by  being  familiar  with  the 
policies,  directives,  and  instructions  which  will  apply.  Anyone  with  review  authority, 
approval  authority,  or  budgeting  authority  in  the  ? oject’s  chain  of  command  can  pose  a 
problem  or  a solution:  it  is  important  to  keep  in  touch  with  their  attitudes  to  discover 
which  may  pose  risks  to  the  project.  Once  the  project  is  underway,  valuable  experience 
becomes  available  to  identify  and  manage  risks:  it  is  essential  to  document  this  experience 
through  engineering  notebooks,  drawings,  progress  reports,  and  memoranda,  especially  il 
personnel  are  constantly  changing,  so  that  this  information  is  not  lost. 

The  risks  associated  with  estimating  may  be  considered  technology -oriented  risks 
even  though  they  may  concern  project  parameters  such  as  cost  and  schedule.  Knowledgeable 
people  and  their  past  experience  may  be  pivotal  in  making  accurate  estimates.  Also,  they 
will  be  essential  in  characterizing  the  accuracy  of  an  estimate  from  any  source.  The  pro- 
gress toward  achieving  all  estimated  parameters  should  be  tracked  through  a testing  program 
(for  technical  parameters)  or  an  appropriate  management  information  system  (for  project 
parameters).  A lack  of  progress  can  be  used  as  a measure  of  risk. 


RISK  CHARACTERIZATION  AND  ASSESSMENT 

When  a nonnegligible  risk  is  identified,  a method  of  managing  it  must  be  formulated: 
this  method  will  depend  on  the  kind  of  risk  and  how  badly  it  can  affect  the  project.  Risk 
elements  may  be  high  or  low.  critical  or  noncritical,  flexible  or  rigid,  dependent  or  inde- 
pendent. and  so  forth  as  described  in  the  introduction  to  this  chapter. 

The  degree  of  risk  may  be  considered  inversely  proportional  to  the  knowledge  ol 
the  associated  task  available  to  the  project.  One  may  assign  subjective  probabilities  of 
success  or  even  develop  statistical  probabilities  in  some  instances.  However,  it  is  normally 
sufficient  to  characterize  the  degree  of  risk  as  follows: 

Low  The  task  has  been  done  before  and  this  experience  is  available  to  the 

project. 

Medium  The  major  portions  of  the  task  have  been  done  before  and  the  exper- 
ience is  available  to  the  project. 

High  One  or  more  major  portions  of  the  task  have  not  been  done  before 

or  the  experience  is  not  available  to  the  project. 

The  principle  relating  knowledge  of  a task  to  the  risk  is  well  established.  However,  il 
the  risk  is  purely  a matter  of  chance,  as  in  a dice  throw,  the  statistical  probabilities  ol 
success/failure  should  be  used.  When  an  estimated  parameter  cost,  schedule,  reliability, 
or  whatever  is  measured  in  the  course  of  the  project,  a measure  ol  risk  may  be  devel- 
oped by  looking  at  the  ratio  of  the  percent  of  goal  attained  to  the  percent  of  resources 
remaining  to  accomplish  the  goal.  If  the  ratio  is  below  a threshold,  say  0.05,  consider 
the  risk  low,  and.  conversely  if  it  is  above  a ceiling,  say  1.05  or  1.10.  consider  the  risk  high. 
For  the  primary  resource  (usually  funds),  this  ratio  will  be  the  actual  expenditure  to  expected 
expenditure  for  the  corresponding  degree  of  progress. 
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The  project  WBS  can  reveal  how  critical  or  potentially  critical  or  noncritical  and 
how  dependent  or  independent  or  interdependent  the  risk  elements  are.  Tools  such  as  the 
PERT/critical  path  method  can  also  help.  No  specific  guidance  can  be  provided  here  be- 
yond stressing  the  importance  of  honestly  evaluating  the  risk  elements  as  they  affect  the 
higher  levels  of  the  WBS  and  the  critical  path  of  the  project.  Each  project  presents  its  own 
difficulties  in  analyzing  task  relationships.  Often  it  will  be  possible  to  develop  numerical 
relationships  by  using  subjective  evaluations;  these  relationships  are  particularly  useful  in 
determining  where  to  expend  risk  management  resources. 

Risk  elements  can  almost  always  be  characterized  as  flexible.  Rigidity  is  normally 
imposed  by  limits  on  resources.  Since  flexibility  and  manageability  are  directly  related,  it 
is  important  to  identify  those  limits  and  to  plan  tasks  to  maintain  flexibility. 


HANDLING  RISK 

The  handling  of  risk  is  probably  the  most  effective  way  to  decide  how  to  allocate 
proje  rces:  after  all.  risk  management  strives  to  maximize  the  chances  of  achieving 

acc>  suits.  The  risk  decisions  can  be  used  to  determine  not  only  how  resources 

sh  ted  but  also  how  the  resources  can  best  be  applied. 

aies  must  be  established.  The  risks  which  bear  the  most  critical  consequences 
Handled  first.  Of  those  critical  risks,  high  risks  should  be  treated  as  top  priority. 
On  a complex  project,  it  is  useful  to  assign  priorities  to  the  risks  and  to  notate  the  WBS  ac- 
cordingly. Table  XX1-1  illustrates  one  such  priority  system. 


TABLE  XXI-1.  RISK  PRIORITIES. 


Risk 


K1SK 

Consequence 

High 

Medium 

Low 

Critical 

1 

3 

Potentially  critical 

4 

5 

6 

Probably  noncritical 

1 

8 

9 

Noncritical 

10 

The  overall  project  risk  can  be  considered  acceptable  if  all  priority  I through  5 risks  can  be 
managed.  When  the  risks  are  not  acceptable,  the  project  should  be  terminated. 

Next,  appropriate  techniques  must  be  selected  to  handle  the  risks.  Table  XXI-2 
lists  some  strategies  and  examples  for  handling  risks.  When  a qualitative  method  has  been 
selected,  an  attempt  should  be  made  to  quantitatively  express  the  risk  situation  as  it  applies 
to  the  project.  This  effort  can  reveal  subtle  interactions  between  project  risks  and  can  in- 
crease confidence  in  decisions  based  on  the  method  even  though  subjectively  derived  prob- 
abilities are  used.  A risk  approach  can  be  modeled  for  virtually  all  major  project  decisions 
including  source  selections.  The  extent  to  which  this  form  of  decision  analysis  is  imple- 
mented should  depend  on  the  scope  of  the  project.  In  many  cases,  these  guidelines  will 
have  served  their  purpose  if  the  project  manager  consciously  considers  risks  along  with  the 
technical,  cost,  and  schedule  factors  of  the  project. 
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TABLE  XXI-2.  RISK  STRATEGIES 


Method  Application  Conditions 

I.  Hedging  Strategies:  Hedging  strategies  offset  risks  by  providing  alternatives  which  have  low  risk  and 
acceptable  results  although  the  ideal  goal  is  only  partially  achieved. 

A.  Parallel-path  method  High,  technology-oriented  risks  Two  or  more  paths  are  pursued 

concurrently;  the  paths  must 
differ  in  risk  characteristic  and 
at  least  one  path  must  be  at  low 
risk.  (Parallel  paths  to  secure 
price  competition  do  not  apply] 

B.  Backup  method  Medium  and  low,  technology-  One  or  more  backup  paths  are 

oriented  risks  identified;  the  paths  must  differ 

in  the  area  of  risk.  For  medium 
risks,  resources  to  implement 
the  backup  are  identified  and 
retained 

II.  Transfer  or  Exchange  Strategies:  Transfer  strategies  exchange  an  unknown  risk  for  a known  accept- 
able risk  or  a known  critical  risk  for  a known  less-critical  risk. 

A.  Warranties  and  Guarantees  Unknown  project-oriented  risks  A known  price  (warranty  cost) 

is  paid  to  fix  a future  condition 
(usually  support  costs);  the  war- 
ranty cost  is  then  budgeted.  The 
unknown  risk  is  supplanted  by  a 
funding  risk. 

B.  Insurance  method  Critical  or  potentially  critical  A known  price  is  paid  to  limit 

project-oriented  risks  possible  future  consequences  as 

in  fire  insurance,  liability  insur- 
ance, etc.  This  method  can  only 
be  applied  where  an  “insurer" 
can  be  found.  Often  this  con- 
dition can  be  satisfied  contrac- 
tually with  a supplier.  The 
price  paid  must  be  within  rea- 
son to  the  condition  insured 
against . 

C.  Contractual  method  Same  as  (A)  and  (B)  When  a contract  is  involved. 

various  contractual  agreements 
can  be  made  to  transfer  risks 
to  the  contractor.  The  con- 
tract must  be  fixed-price  and 
negative  price  incentives  may 
be  indicated.  ? 

III.  Reduction  Strategies:  Reduction  strategies  amount  to  better  management  and  to  careful  control, of 
resources  which  lead  to  a higher  probability  of  success.  They  may  be  applied  to  virtually  any  risk 
situation  including  in  combination  with  other  strategies.  To  reduce  technical  risk,  top  personnel 
would  be  assigned  to  the  task.  Cost  and  schedule  risks  would  require  more  intensive  preplanning  and 
effective  management  controls  and  communications.  In  the  application  of  reduction  strategies,  care 
must  be  taken  not  to  create  risks  on  other  tasks  by  stripping  them  of  resources  and  not  to  interfere 
with  the  task  through  “micromanagement."  If  problems  do  occur,  the  goal  is  to  minimize  their  effect. 
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I AH  LI  XX1-2  (Continued) 


IV.  Avoidance  Strategies:  Avoidance  strategies  are  alternate  paths  which  avoid  exposure  to  the  parti- 
cular risk.  Primarily,  avoidance  strategies  are  applied  to  technology-oriented  risks.  I samples  include 
the  choice  of  one  technical  approach  over  another  because  of  known  past  difficulties  with  the  latter. 
A pseudo  technology -oriented  political  risk  might  be  avoided  by  structuring  the  proiect  loi  sponsor- 
ship by  nonhostile  offices  or  to  bypass  a hosii'e  office  in  the  approval  cycle.  The  crux  of  any  avoid- 
ance strategy  is  knowing  that  a risk  exists  and  devising  a means  of  nonexposure. 

V.  hvasion  Strategies:  hvasion  strategies  utilize  prior  planning  to  avoid  risk  consequences  even  though 

a failure  occurs.  Because  of  legal  and  technical  restraints,  evasion  strategies  are  almost  wholly  limited 
to  political  risks.  Usually  this  requires  developing  a "power  base'  of  influential  "friends  of  the  pro- 
ject”: occasionally  the  "snow  job"  is  a viable  tool.  The  ultimate  success  of  an  evasion  strategy  in  a 
technical  project  application  relies  on  honest  facts  and  the  apparent  ultimate  achievement  of  the 
goal. 

VI  Risk  Assumption:  Assumption  strategies  recognize  the  unlikelihood  of  the  expected.  Applied  to 
project-oriented  risks,  especially  funding  and  scheduling,  assumption  strategies  plan  "padding"  into 
project  estimates;  the  key  to  successful  risk  assumption  is  limiting  the  extent  of  the  padding  to  what 
is  sufficient  while  ensuring  an  adequate  amount.  Applied  to  technology -oriented  risks,  assumption 
strategies  tend  toward  ultraconservative  design  parameters.  Care  must  be  taken  not  to  misapply 
such  strategies,  especially  to  technical  problems,  since  they  tend  to  produce  gross  overspecification 
with  drastically  increased  costs.  Risk  assumption  must  not  become  a “cover  your  anatomy"  ploy 

BETTER  ESTIMATES  USING  RISK  ASSESSMENTS 

Ati  estimate  is  a judgment  as  to  what  is  most  likely:  that  is,  it  is  an  expected 
value.  When  estimates  are  made  at  a low  level  in  the  WBS.  the  combined  estimate  at  the 
top  level  is  more  accurate  because  there  is  less  chance  of  omitting  key  elements'  and  be- 
cause there  is  a statistical  tendency  toward  eliminating  the  collective  error  in  the  esti- 
mates. However,  a simple  combination  of  estimates  yields  the  most  expected  value  for 
the  estimated  parameter  disregarding  the  effects  of  interdependent  and  dependent  tasks. 
Because  of  task  dependencies,  an  overrun  in  a task  can  be  propagated  to  other  tasks 
whereas  an  underrun  can  remain  isolated:  therefore,  a “best  estimate"  will  almost  always 
be  overrun.  A subjective  risk  assessment  technique  can  be  applied  to  correct  higher-level 
estimates  to  yield  a much  more  precise  estimate. 

As  an  example,  let  us  assume  that  a work  package  consists  of  n tasks  for  which 
a parameter  is  being  estimated  in  an  amount  an.  The  parameter  may  be  cost,  time  to 
completion,  or  any  other  expended  resource.  Figure  XXI- 1 shows  the  traditional  esti- 
mate for  the  work  package. 


Figure  XXI- 1 . The  traditional  estimate. 
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venerate  a probability  ot  success  lor  each  task  and  designate  it  I’d;  although  the  proba- 
bility is  subjective,  the  best  estimate  ol  the  knowledgeable  estimator  of  an  will  be  a suf- 
ficiently accurate  P„.  Flu  re  is  now  an  identified  risk  of  l-P,,.  Should  a failure  occur,  ad- 
ditional resource  will  be  utilized  to  attempt  to  complete  the  task,  this  effort  will  also  have 
a probability  ol  success.  This  additional  effort  is  a conditional  task  n'  Figure  XXI-2  il- 
lustrates the  combined  work  package  estimate. 
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Figure  XXI-2.  An  estimate  with  risks  assessed. 


Any  number  ol  reiterations  may  be  accounted  for  until  a sufficiently  high  probability 
ol  success  is  reached.  In  the  illustration,  task  3 has  a second  conditional  task  n”  which 
depends  upon  the  failure  of  the  first  conditional  task  n';  on  the  other  hand,  task  4 is 
sufficiently  certain  that  no  condition  task  is  estimated.  The  technique  is  surprisingly 
easy  to  use  and  becomes  easier  with  experience.  In  conjunction  with  standard  estimat- 
ing techniques,  this  method  produces  estimates  which  are  approximately  an  order  of 
magnitude  more  accurate  lor  complex  projects.  If  the  project  is  using  a computerized 
WBS.  this  method  can  be  integrated  into  the  program. 
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EMX  MANAGEMENT 


One  of  the  most  severe  environments  encountered  by  equipments  in  military  applica- 
tions is  the  electromagnetic  (EM)  environment.  The  severity  of  military  EM  environments 
tends  to  grossly  exceed  that  of  otherwise  comparable  commercial  EM  environments:  therefore, 
an  effective  program  of  EM  management.  ■■  sessment.  and  engineering  is  required  to  assure 
system  effectiveness.  This  need  extends  equally  to  off-shelf  equipments  and  to  equipment 
developments,  although  the  actions  taken  are  affected  by  the  equipment  source. 

Problems  arising  from  EM  sources  are  of  three  types: 

• Unwanted  emissions  from  the  equipment 

• Damage  to  the  equipment  by  the  EM  environment 

• Undesirable  modification  to  equipment  performance  due  to  emissions  from  other 
equipments 

The  damaging  effects  of  these  problems  fall  into  one  or  more  of  the  following 
categories: 


• Hazards  to  personnel 

• Hazards  to  ordnance 

• Hazards  to  other  equipments 

• Degraded  equipment  performance 

• Degraded  security 

The  engineering  tools  which  are  available  to  combat  EM  problems  include: 

• Filtering 

• Shielding 

• Grounding 

• Application  of  special  technology 

• Careful  component  selection  and  application 

• Installation  control 

Each  of  these  tools  costs  money  to  implement:  how  much  money  depends  on  the 
degree  of  protection  needed  and  on  the  point  at  which  protective  measures  are  applied  in  the 
design  ot  the  equipment.  When  design  changes  are  mandated,  they  are  always  more  expen- 
sive than  proper  initial  design;  furthermore,  design  changes  are  always  more  extensive  as  the 
equipment  design  matures.  The  variety  and  impact  of  the  problems  and  prospective  solutions 
demand  management  attention  in  order  to  effect  an  acceptable  product  and  to  efficiently 
utilize  project  resources. 

There  are  a number  of  terms  which  are  somewhat  peculiar  to  the  I \1  field: 

EMC  Electromagnetic  compatibility  a term  encompassing  the  issues  of  an 

equipment  operating  properly  within  its  EM  environment  and  not 
causing  interference  to  other  equipments 
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A 


EMI 

EMP 

HMX 

HERO 

RADHAZ 

TEMPEST 


Electromagnetic  interference  negative  effects  ot  one  equipment  on 
another 

Electromagnetic  pulse  an  EM  effect  ot  a nuclear  blast 
A blanket  term  covering  all  EM  related  problems 

Hazardous  electromagnetic  radiation  to  ordnance  (see  M1L-STD-I377) 
Electromagnetic  radiation  hazard  to  personnel 
A term  covering  security  problems  which  are  EM  design  oriented 


This  list  is  not  complete  but  is  sufficient  for  the  discussion  which  follows. 

Hazards  to  personnel  include  shock  hazards  and  RADHAZ.  The  approaches  toward 
reducing  these  hazards  include  safety  interlocks,  protective  enclosures  around  high-hazard 
sections,  good  grounding  techniques,  and  warning  labels  and  signs.  Requirement  1 of  MIL- 
STD-454  covers  these  approaches.  No  equipment  or  system  should  be  exempted  from  these 
provisions. 

Hazards  to  ordnance  are  normally  treated  by  control  of  the  ordnance  design  and  by 
ordnance  handling  procedures  which  require  shutting  down  all  emitters  in  the  2-32-MHz  band 
(the  frequencies  of  greatest  ordnance  susceptibility).  The  equipments  which  may  be  connected 
to  ordnance,  directly  or  indirectly,  must  be  designed  to  protect  sensitive  circuits  (particularly 
firing  circuits)  from  the  EM  environment.  The  steps  required  include  proper  grounding 
techniques,  shielding,  filtering  against  transients  and  spurious  signals,  and  designs  to  preclude 
false  signaling;  safety  interlocks  and  fail-safe  designs  should  be  integral  to  all  equipments 
connected  to  ordnance. 

If  an  equipment  will  be  operating  remotely  and  independently  trom  any  other  equip- 
ment, EM1/EMC  is  not  a particular  problem;  however,  most  applications  will  require  shared 
power  systems,  antenna  systems,  control  systems,  or  installation  spaces.  Usually,  EMC’  will 
be  important.  Figure  V-3  shows  the  relationship  between  the  signiticant  F.MI  and  EMC 
specifications  and  standards.  "'Suggestions  for  Designers  ol  Navy  Electronic  Equipment' 

( 1075  Edition,  NELC/TD  300)  contains  many  tips  on  good  EM  engineering  techniques.  The 
project  manager  should  ensure  that  lead  engineers  are  familiar  with  EM  design  practices  and 
that  the  test  program  incorporates  EM  testing.  Ml  I. -STD-40 1 is  the  primary  EMI  require- 
ments standard.  It  describes  the  “normal  worst  case”  EM  environment;  as  such,  it  over- 
specifies some  applications  and  underspecities  others.  Where  more  precise  validated  informa- 
tion is  available  for  a particular  application,  it  should  be  used  in  lieu  ol  Mil  -S  I D-4(> I . For 
general  applications  or  other  applications  in  which  a large  number  ot  equipments  are 
collocated  and  are  subject  to  changes  in  configuration,  then  apply  the  more  stringent  require- 
ments. The  test  requirements  of  M1E-S  1 D-4b2  are  valid  even  if  the  test  parameters  are 
modified  by  better  information.  However,  the  broadband  tests  of  CEO  I and  the  requirements 
of  CE05  are  difficult,  if  not  impossible,  to  test  in  a valid  manner;  they  should  be  excluded 
from  project  test  requirements.  Particular  attention  should  be  paid  to  spiuious  emissions. 

Off-shelf  equipments  should  be  tested  during  the  screening  process  to  assure  compli- 
ance to  EM  requirements.  Because  of  the  peculiarly  stringent  military  EM  environments, 
many  off-shelf  commercial  equipments  will  require  filters  and  careful  installation  control  to 
assure  adequate  compatibility.  Some  commercial  equipments  will  show  susceptibility  to 
damage  in  military  EM  environments  and  should  be  disqualified  if  the  damaging  conditions 
occur  during  normal  operations  or  frequently  and  if  adequate  protection  cannot  be  added 
economically. 

I n the  conduct  of  an  EMC  program.  Mil  -I IDBk-235.  Mil  -I IDBK-237.  and  Mil  -IIDBK 
23S  should  prove  useful.  Mil  -IIDBk-237  discusses  I NK  program  management. 
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MIL-HDBK-235  describes  specific  EM  environments.  MIL-HDBK-238  describes  numerous 
radiation  hazards  and  ways  to  handle  them.  Also,  MIL-HDBK-241  provides  guidance  for 
designing  power  supplies  and  filters  for  EMC. 

EMP  is  a very  difficult  problem  and  an  expensive  one  to  solve.  Because  it  is  so  expen- 
sive to  treat,  EMP  management  should  be  applied  only  to  vital  systems.  Consider  all  systems 
wiped  out  by  EMP  damage  except  your  system  and  EMP  qualified  systems:  if  your  system  is 
still  essential  to  critical  operational  missions,  then  it  is  a candidate  for  EMP.  All  the  EMX 
tools  must  be  brought  into  play  to  attack  EMP:  therefore,  it  is  essential  that  design  personnel 
be  familiar  with  the  EMP  problem  and  approaches  to  its  solution.  Courses  are  available  on 
EMP  which  should  be  attended  by  all  designers.  Take  a common  sense  approach  to  the 
problem.  Must  the  system  operate  flawlessly  during  the  EMP,  or  can  some  downtime  be 
tolerated?  Specially  designed  microcircuits  and  semiconductors  may  be  required  to  survive 
EMP.  EMP  is  characterized  by  extremely  fast  rise  times  and  very  broadband  high-intensity 
Fields;  both  effects  must  be  considered.  Fast-acting  protective  devices  may  be  indicated. 
Another  approach  is  to  design  the  equipment  so  that  EMP  damage  will  be  confined  to  a 
known  component  or  module,  then  a spare  can  be  designed  into  the  system,  but  out  of  the 
circuit,  to  provide  a low  downtime.  Include  EMP  testing  in  your  program.  Knowledgeable 
personnel  and  early  design  attention  are  essential  to  efficient  EMP  management. 

TEMPEST  is  similar  to  EMI  in  its  basic  manifestations,  causes,  and  cures.  However. 
TEMPEST  is  concerned  with  signal  intelligence  whereas  EMI  is  not:  this  is  reflected  in  the 
respective  testing  philosophy.  The  power  and  frequency  ranges  at  which  intelligence  can  be 
recovered  are,  in  general,  more  stringent  than  those  which  cause  interference.  TEMPEST  re- 
quires early  consideration  of  knowledgeable  design  practice.  The  standards  for  TEMPEST 
are  NACSEM  5 1 00  and  NAC'SEM  5 1 03.  Because  it  is  a specialized  engineering  discipline, 
project  managers  for  equipment  which  handles  classified  information  should  request  guidance, 
technical  assistance,  and  publications  from  the  Nava!  Electronic  Systems  Security  Engineering 
( enter  (NESSEX  ).  3801  Nebraska  Avenue  NW,  Washington.  DC  20300.  Commands  and 
activities  of  the  Naval  Material  Command  should  send  requests  directly  to  the  Commanding 
Officer.  NESSEC.  Industrial  organizations  performing  on  contract  should  submit  requests 
through  the  contracting  officer  and  the  sponsoring  activity.  Other  naval  commands  and 
activities,  and  non-Navy  agencies,  may  contact  NESSEC  for  similar  assistance.  It  is  essential 
that  this  contact  be  established  early  to  assure  adequate  planning,  funding,  and  execution  of 
TEMPEST  related  tasks. 

When  installation  control  is  to  be  employed,  as  it  must  be  in  secure  systems  and  EMP 
protected  systems,  the  control  requirements  must  be  cited  in  the  installation  planning  docu- 
mentation (see  chapter  XVIII ).  Special  installation  controls  will  involve  additional  design, 
planning,  and  supervision  by  the  installing  activity.  Wherever  possible,  it  is  advisable  to  build 
features  into  the  equipment  which  force  proper  installation  to  avoid  installation  control 
problems:  however,  systems  with  interconnecting  cabling  to  units  in  remote  locations  must 
contend  with  the  problems  of  installation  control. 

Standards  for  bonding  and  grounding  techniques  are  found  in  Mil  -SI  D- 1310  and 
MIE-B-5087. 


PARTS  MANAGEMENT 

I he  management  of  parts  selection  and  application  is  essential  to  the  success  of  an\ 
engineering  project.  Unlike  other  management  tasks,  parts  management  is  not  subject  to  the 
rigorously  well  defined  guidance  that  are  other  tasks,  such  as  contiguration  management. 
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Parts  management  policy  must  he  set  hy  the  project  manager  to  conform  to  the  conditions 
which  exist  for  that  project:  the  policy  is  implemented  by  the  designers.  In  this  context,  a 
part  must  be  considered  any  piece  part,  module,  subassembly,  assembly,  or  unit  which  might 
he  stocked  as  an  item  of  supply  support  or  the  components  making  up  such  an  item.  Obvi- 
ously the  parts  management  policy  must  eventually  dictate  the  provisioning  decisions  for  an 
equipment,  since  the  policy  dictates  the  design  which  must  he  provisioned.  Past  attempts  to 
mandate  a general  parts  management  policy  have  amounted  to  a blanket  standard  piece  parts 
standard  module  policy  which  has  not  always  been  consistent  with  individual  project  goals: 
ie,  utilization  of  commercial  equipment,  specialized  technologies,  or  low-cost  high-availability 
parts  or  attainment  of  reliability  goals.  A much  broader  policy  of  standardization  is  enun- 
ciated by  NAVMA  I I NS  I'  4 I 20. l>7  (series)  which  is  summarized  in  chapter  \IV  in  the  section 
titled  Standardization  of  Components  Tquipment  ((  I ).  lite  l.ow  Cost  l-.lectronics  study 
shows  that  the  provisions  of  this  instruction  are  largely  ignored  or  inappropriately  applied 
because  of  a lack  of  good  technical  guidance  to  project 'acquisition  managers,  l his  section 
will  attempt  to  provide  such  guidance. 

In  general,  five  factors  must  be  considered  in  setting  a parts  management  polio  : 

Performance 

Reliability 

Maintenance 

Provisioning 

Cost 

In  addition,  the  project  manager  must  consider  parts  availability,  since  delays  in  parts 
deliveries  may  have  a very  negative  el  feet  on  the  project  schedule.  I Itese  (actors,  as  they 
interact  with  each  other,  may  be  lumped  into  three  categories  of  issues: 

Standardization 

I Effectiveness 
[•Efficiency 

STANDARDIZATION 

There  are  two  types  of  standardization  intrasyslem  and  inlersystem.  The  goals  ol 
intrasystem  standardization  are  to: 

• Increase  the  producibilitv  of  the  system  by  reducing  the  number  of  different  kinds 
of  components 

• Increase  system  supportahility  through  widespread  intrasyslem  commonality  of 
designs,  modules,  etc 

• Decrease  system  documentation  costs  through  commonality  of  designs,  modules, 
etc 

• Create  a situation  in  which  all  items  provisioned  can  be  ordered  in  economically 
large  quantities  (see  tig  VI  I) 

• Decrease  system  downtime  for  parts  (see  fig  1V-4) 
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• Decrease  system  design  time  by  limiting  the  choices  available  to  the  designer 

• Establish  standard  intrasystem  interlaces 

Most  simply,  intrasystem  standardization  strives  to  make  the  widest  possible  use  of  the  fewest 
possible  different  kinds  of  parts  and  designs.  I'he  greatest  advantages  of  such  programs  as  the 
Standard  Electronics  Module  (SIM)  and  the  Standard  Hardware  Program  (SUP)  lie  in  the 
ready  availability  through  them  of  design  building  blocks  for  application  in  intrasystem 
standardization  efforts,  file  goals  of  intersvstem  standardization  are  to  minimize  the  num- 
ber of  different  logistics  items  which  must  be  supported  and  to  maximize  interchangeability 
between  items  of  like  functions. 

The  following  steps  are  recommended  to  implement  the  standardization  objectives  of 
parts  management : 

1 . Establish  a project  policy  of  maximizing  intrasystem  standardization. 

2.  Wherever  possible,  make  use  of  existing  standards  (standard  interfaces,  standard 
modules,  standard  parts,  standard  equipments,  standard  test  provisions,  etc): 
however,  do  not  enforce  this  provision  to  the  degradation  of  other  project 
requirements.  Established  industry  standards  should  be  considered  as  well  as 
military  standards. 

3.  Minimize  system-peculiar  designs  and  components. 

4.  In  selecting  parts,  choose  preferred  parts  (MIL. -STD-2421  over  other  standard 
parts  (controlled  by  military  specification),  standard  parts  over  controlled  parts 
(nonstandard  parts  already  supported  in  the  National  Slock  System),  and  con- 
trolled parts  over  all  other  parts.  This  step  is  implemented  in  accordance  with 
Ml  L-STD-l  <>3  I and  Mil  -STD-143. 

5.  Where  other  factors  (such  as  cost  or  availability)  militate  against  the  use  of  the 
part  which  would  otherwise  be  selected  under  step  3.  select  a part  which  can  be 
replaced  by  the  step  3 selection  for  repair  provisioning  purposes  and  show  the 
step  3 selection  in  the  provisioning  documentation.  This  step  is  implemented  in 
accordance  with  Requirement  7 of  Ml  1 -STD-454. 

Document  all  system  interfaces,  down  to  the  level  of  standardization,  using 
functional  specifications  as  discussed  in  chapter  VI. 

Step  (>  is  particularly  important,  since  it  establishes  the  mechanism  for  future  design  evolution 
within  the  framework  of  standardization:  figure  WII-I  illustrates  this  mechanism. 

EFFECTIVENESS 

Parts  management  for  overall  system  effectiveness  is  most  often  cloaked  as  "good 
engineering  practice.”  I'he  performance  of  the  system  hinges  on  proper  parts  selection  and 
application.  I he  reliability  of  the  system  depends  on  ( I ) the  inherent  failure  rate  of  the 
selected  parts  and  ( 2)  the  stresses  put  on  the  parts  by  their  design  application.  I he  availa- 
bility of  the  system  depends  on  its  achieved  reliability  and  the  time  to  restore  il  to  operation 
when  a failure  occurs.  I he  availability  of  parts  is  a major  factor  in  the  ability  to  repair  a sys- 
tem: therefore,  consideration  of  both  current  and  future  part  availability  is  warranted.  Un- 
doubtedly. performance  factors  play  a major  role  in  part  selection:  a partial  list  of  these  fac- 
tors would  include  size,  weight,  form  factor,  power  consumption,  environmental  ratings. 
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spec  xxx 


IB) 


1C) 

SPEC  00  XXX  A 


ID) 

SPEC  XXX  B 


SPEC  YYY 


SYSTEM  A 
SYSTEM  B 


SYSTEMS  A,  B.  C 
SYSTEM  D 


SYSTEM  E 
SYSTEM  F 


SYSTEM  C 


SYSTEM  G 


lA)  ITEM  A IS  DEVELOPED  FOR  SYSTEM  A AND  DOCUMENTED  BY  SPEC  XXX 

IB)  ADDITIONAL  USE  FOR  ITEM  A IS  FOUND  IN  SYSTEM  B AND  SYSTEM  C. 

(C)  TECHNOLOGY  ADVANCES  LEAD  TO  ITEM  A'.  CAN  ALSO  BE  UT  ILIZED  IN  SYSTEM  D, 
WHICH  REQUIRES  THE  IMPROVED  CHARACTERISTICS.  ITEM  A IS  DOCUMENTED  BY 
SPEC  00  XXX  A 

ID)  A NEW  GENERATION  OF  TECHNOLOGY  LEADS  TO  ITEM  B WHICH  IS  DEVELOPED  FOR 
SYSTEM  E AND  DOCUMENTED  BY  SPEC  YYY,  ITEM  B IS  FUNCTIONALLY  LIKE  ITEM  A' 
BUT  SIGNIFICANTLY  SMALLER  IN  FORM  FACTOR  AND  LESS  EXPENSIVE.  SO  SPEC  XXXB 
IS  DEVELOPED  TO  ADAPT  ITEM  B TO  SYSTEMS  A,  B.  C,  AND  D AND  TO  SUPERSEDE 
SPEC  XXX  AND  SPEC  OOXXX  A.  LATER.  NEW  APPLICATIONS  FOR  ITEM  B ARE  FOUND 
IN  SYSTEMS  F AND  G. 


Furore  WIFI,  Growth  ol  a standard. 

oiul  tolerance.  "Suggest ions  tor  Designers  ol  Navy  I lectronic  I quipment  I l‘*  ^ edition. 

\!  I ( l'l)  .W01  provides  many  points  to  aid  in  avoiding  pit  tails  in  part  selection:  the 
appropriate  military  specifications  and  "selection  and  use  standards  arc  also  1 1 sc  till.  Beyond 
these  standard  lactors.  parts  should  he  selected  as  follows: 

I Select  high  reliability  parts. 

’ I \e  parts  well  within  their  rated  limits:  derate  the  parts  lor  the  environment 
lhe\  must  endure  in  service  ( temperature.  I Ml.  vibration!. 

; ( hoose  parts  whose  dominant  failure  mode  has  minimum  effect  on  the 

equipment. 

1 Select  parts  and  designs  which  do  not  require  other  components  to  correct  their 
deficiencies  i for  instance,  vihraloi  ty  pe  power  supplies  will  normally  require 
filtering  to  take  out  the  I Ml  they  produce). 

I ake  into  account  part  tolerances  and  value  changes  undei  stress  and  aging. 

i,  t hoo\e  parts  which  are  produced  by  mature,  large-volume  manutacturing 
processes  where  other  factors  ( si/e.  weight,  speed,  etc)  do  not  dictate  a less 
mature  technology . 

\v out  sole  source  and  proprietary  parts. 

v i onlorm  to  standardization  steps  4 and  5 When  specialized  screening  is  indi- 
i ated.  cite  the  screening  requirements  w Inch  are  in  excess  ol  high  reliability 
standard  parts  qiiahlicution  requirements 
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Cost  ami  availability  of  parts  arc  factors  which  must  always  he  weighed  in  part  man- 
agement decisions.  However,  these  factors  should  not  he  given  primacy  over  other  part  selec- 
tion factors  since  the  initial  cost  of  components  is  only  a small  portion  of  an  equipment's 
cost  (typically  10  for  military  electronics)  and  supposed  savings  are  quickly  obscured  by 
high  support  costs.  Nevertheless,  a number  of  alternatives  may  be  available  to  the  project  in 
its  parts  decisions. 

1.  In  more  complex  parts  (assemblies  and  units),  off-shelf  items  should  be  consid- 
ered in  preference  to  developing  a new  item. 

2.  Manufacturers’  high-reliability  lines  of  commercial  parts  are  often  much  more 
readily  available  than  military  parts  and  are  usually  less  expensive;  these  parts 
can  be  used  to  advantage  in  design  and  even  in  limited  production  as  long  as  the 
military  standard  part  is  used  as  the  provisioning  part  (see  standardization 

step  5 ). 

High  reliabilities  meeting  or  exceeding  military  standards  are  often  attainable  by 
applying  an  appropriate  screen  to  commercial  parts;  this  applies  to  piece  parts 
and  whole  equipments  alike.  In  applying  screens  to  piece  parts,  large  enough 
volumes  of  parts  must  be  screened  to  amortize  the  screening  setup  costs  and  the 
costs  of  rejected  parts.  When  a h igh-rel iabil 1 1 x standard  part  exists,  it  should  be 
specified  in  provisioning  documentation. 

4.  When  parts-peculiar  are  justified  (such  as  custom  LSI),  consideration  should  be 
given  to  total  life-cycle  procurement  techniques  to  preclude  uneconomical 
small-lot  reprocurements.  (A  total  life-cycle  procurement  combines  initial  re- 
quirements and  all  projected  support  requirements.) 

( ONI  It. I RA  I ION  MAN  UiLMLNT 

I lie  purpose  of  configuration  management  (CM)  is  to  ensure  that  the  knowledge  re- 
quired to  design,  operate,  reproduce,  improve,  and  support  a system  is  retained.  I'his  knowl- 
edge is  initially  in  the  minds  of  the  designers,  but  eventually  it  must  be  documented  and 
controlled.  During  design,  task  leaders  should  be  held  responsible  for  ensuring  that  the 
proper  documentation  is  created  and  maintained  accurately;  this  is  a function  of  data  manage- 
ment (see  chapter  \\  and  NAVMA  I INS  I 4000.15  ( series)).  Once  the  documentation  has 
been  produced,  verified,  and  accepted,  it  becomes  subject  to  the  more  formal  controls  of 
configuration  management  to  ensure  that  the  equipment  and  its  descriptive  data  are  indeed 
of  t he  same  design. 

Configuration  management  should  be  executed  in  accordance  with  NAVMA  I INS  I 
4 I 50. 1 (series).  CM  is  implemented  through  three  information  baselines: 

Lunctional  baseline  based  on  the  system  specification 

Allocated  baseline  based  on  detailed  system  partitioning 

Product  baseline  based  on  the  established  system  design 

I he  functional  and  allocated  baselines  consist  of  total  functional  descriptions  of  the  identi 
fieri  configuration  items.  I lie  product  baseline  consists  ot  the  total  design  description  ol  all 
system  items  down  to.  and  including,  the  level  of  standardization  (see  chapter  VI ).  this  will 
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normally  consist  of  a mixture  of  functional  and  fabrication  specifications,  drawings,  and 
associated  lists.  Once  a baseline  is  established,  it  can  only  be  changed  through  an  engineering 
change  proposal  t b.C  Pt.  rwo  classes  ot  LC  P and  defined  by  MIL-S  I D-480  ( lass  I and  ( lass 
II.  c lass  I changes  must  be  justified  on  tfie  basis  of  their  beneficial  eflecl  fo  the  configura- 
tion item  and  must  be  approved  by  the  Government  Change  Control  Hoard  (CCB)  prior  to 
implementation  (this  CCB  w.ill  reside  internally  to  the  project  until  the  production  deploy- 
ment phase:  then  the  CCB  of  the  cognizant  systems  command  will  take  over).  Class  II  changes 
are  justified  only  on  the  basis  of  their  benefits  to  the  originator  or  the  government  and  may 
be  implemented  as  long  as  they  will  not  be  detrimental  to  the  government  and  when  the 
government  concurs  that  the  E(2P  meets  this  criterion.  In  the  implementation  of  CM,  the 
following  recommendations  are  made: 

1.  Implement  CM  to  the  level  of  standardization:*  do  not  require  CM  controls 
below  that  level. 

2.  Initially  implement  CM  controls  to  the  level  of  repair  until  field  reliability  objec- 
tives have  been  met:  then  extend  the  level  of  control  as  necessary. 

3.  Defer  invocation  of  CM  controls  on  items  under  a long-term  warranty  until  the 
warranty  period  is  about  to  end  and  the  government  is  about  to  take  over 
maintenance  from  the  warrantor. 

4.  Defer  full  spares  stocking  until  after  the  product  baseline  is  established  at  the 
level  of  standardization. 

5.  Determine  the  nondetrimental  condition  of  Class  II  changes  on  the  basis  ot  both 
performance  and  total  life  cost. 

Two  additional  instructions  cover  additional  CM  policy  for  tactical  system  software 
(NAVMA  I INST  4 1 30.2)  and  A DP  hardware  and  software  subject  to  joint  CM  (NAVMA  I - 
INST  41  30. 3 A).  Project  managers  of  systems  incorporating  software  should  be  aware  of  the 
provisions  of  these  instructions. 

Other  references  include: 

Mil  -STD-48 1 A 
Ml  1.-STD-482A 
Mil  .-STD-483 
MIL-STD-1456 
Mil -STD-1521 

QUALITY  MANAGEMENT 

Quality  is  defined  as  the  composite  of  all  the  attributes  or  characteristics,  including 
performance,  of  an  item  or  product  (source:  Dol)  Directive  4155.11).  I here  are  two  major 
facets  of  quality  management  quality  assurance  (QA)  and  quality  control  (Q(  ).  QA  is  a 
planned  and  systematic  pattern  of  all  actions  necessary  to  provide  adequate  confidence  that 
an  item  or  product  conforms  to  established  technical  requirements:  Q(  is  the  collection  >4 

* I he  level  ol  standardization,  as  debited  in  chapter  VI . will  include  the  lower  level  of  complexity  ol  1 1 ) the 
level  ol  (organic  maintenance)  repair,  or  ( ?)  the  level  just  below  the  level  ot  design  ownership:  litis  will 
incorporate  requirements  lor  intermediate  and  depot  level  repair  under  government  cognizance. 
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actions  taken  for  the  purpose  of  preventing  production  ol  detective  materials  or  items 
(source:  MII.-STD-109).  Once  technical  requirements  have  been  derived  from  operational 
requirements,  QA  becomes  an  unavoidable  management  function;  however,  the  imposition 
of  formal  QA  procedures  will  depend  upon  individual  project  requirements. 

Categorize  each  technical  requirement  as  either  an  attribute  (it  exists  or  it  does  not 
exist)  or  a characteristic  (it  exists  in  various  degrees);  further  identify  whether  the  require- 
ment is  critical  (unacceptabilitv  is  likely  to  result  in  a hazardous  or  unsafe  condition  tor 
individuals  using,  maintaining,  or  depending  on  the  product  or  is  likely  to  prevent  perlorm- 
ance  of  the  tactical  mission),  major  (unacceptability  is  likely  to  result  in  failure  ot  the  product 
for  its  intended  purpose),  or  minor  (unacceptability  is  unlikely  to  attect  the  usability  ot  the 
product  for  its  intended  purpose).  Minimum  acceptable  standards  should  be  established  tor 
each  characteristic  as  well  as  desired  levels  ot  performance.  I he  Section  4 provisions  ol  the 
system  specification  should  fully  detail  the  methods  of  inspection  and  test  which  will  be 
implemented  to  assure  the  adequacy  of  the  final  product. 

To  simplify  the  quality  management  problem,  decide  what  attributes  or  characteris- 
tics are  products  of  pure  design  effort  or  subject  to  the  vagaries  ot  the  production  process. 

Of  those  which  are  affected  by  production  factors,  separate  out  the  critical  or  highly  variable 
attributes  and  characteristics.  Design  features  and  many  minor  requirements  can  be  satis- 
factorily inspected  on  a one-time  basis.  Critical  and  highly  variable  major  features  should  be 
subjected  to  100' 7 inspection;  other  production-affected  features  can  be  subjected  to 
sampling  inspections  (see  M1L.-STD-105  (attributes)  and  MIL-SI  D-414  (variables)).  It  should 
be  remembered  that  there  are  many  components  to  quality  performance,  reliability,  md 
workmanship,  for  instance.  An  item  can  have  high  quality  overall  and  still  be  deficient  in  any 
one  area  (like  reliability).  It  is  important  to  structure  the  accept  reject  criteria  in  a manner 
such  that  an  unacceptable  determination  in  any  critical  or  major  teature  results  in  a rejection. 
Do  ik'*  inspect  or  test  for  features  which  are  not  requirements;  inspection  lor  cosmetic 
purposes  only  drives  up  costs. 

The  following  references  may  be  useful  in  imposing  QA  QC  requirements  on 
contractors: 

Topic 

Ml L-Q-985XA  Quality  Program  Requirements 

M1L-I-45208A  Inspection  System  Requirements 

MIL-C-45662A  Calibration  System  Requirements 

M1L-S-52779  Software  QA  Program  Requirements 

MIL-STD-252B  Workmanship  Standards 

Ml  l -S  I D- 1 535 A Supplier  QA  Program  Requirements 

Four  rules  of  analyzing  quality  cost  data: 

1 . If  number  of  dollars  spent  on  corrective  action,  quality  audits,  etc.  shows  de- 
crease in  scrap-rework  and  other  deviation  costs  > cost  ol  quality  activities: 
continue  spending  otherwise  don't. 

2.  Publicize  the  results  of  quality  cost  data  throughout  the  project. 

4.  Be  as  careful  of  low  expenditures  for  quality  as  high. 

4.  Spend  money  where  it  counts.  In  general,  it  product  is  SO  percent  ot  budget, 
spend  most  quality  money  there. 
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Checklist  for  Quality  Cost  Accounts 


Quality  Creation 

1.  Vendor  control 

2.  Planning  and  implementing  test:  inspect  and  process  control  procedure 

3.  Design  review 

4.  Training  and  education 

5.  Review  of  material  handling  and  packaging 

Quality  Maintenance 

1 . Calibration  and  repair  of  TE 

2.  Field  testing  of  products 

3.  Procedures  audit  1 

4.  Failure  analysis 

5.  Data  maintenance 

ts.  Audit  of  corrective  action  1 

Internal  Quality 

1 . Scrap 

2.  Rework 

3.  1 00'.'  sorting  of  suspected  material 

4.  Material  review  board  activities 

5.  Production  facilities  downtime  due  to  re-setups 

(>.  Extra  vendor  advice 

External  Quality 

1 . Field  complaints 

2.  Customer  allowance  for  rejections 

3.  Product  serv  ice  and  repairs 
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Under  the  present  system  of  hardware  acquisition,  all  equipments  which  are 
designated  for  Navy  shipboard  use  are  supposed  to  be  subjected  to  the  analyses  and 
testing  rigors  specified  in  MIL-E-16400.  A review  of  the  requirements  of  M1L-E- 
16400  in  relation  to  measured  ship  oard  data  indicates  that  savings  in  time  and  money 
can  be  gained  by  reducing  environmental  analyses  and  requirements  testing.  Before 
proceeding  further,  it  should  be  stated  and  emphasized  that  this  study  does  not  apply 
to  critical  equipment  (critical  equipments  being  those  without  which  the  ship  cannot 
operate  or  perform  its  mission).  Any  reduction  ot  environmental  requirements  should 
be  based  upon  the  elimination  or  modification  of  those  tests  and  analysis  requirements 
pertaining  to  shipboard  conditions  deemed  to  be  abnormal;  that  is,  conditions  occurring 
very  rarely  or  never  having  occurred  to  date  to  an  operational  Navy  ship.  Examples 
of  such  conditions  would  be  a nearby  underwater  blast,  nuclear  air  blast,  and  simultane- 
ous high  temperature  and  high  relative  humidity.  The  sections  ot  M I L-E- 1 6400  des- 
cribing environmental  tests  and  analyses  were  reviewed  to  determine  ( 1 ) which  would 
be  classed  as  abnormal  occurrences  not  applicable  to  noncritical  equipments  and  which 
could  be  modified  to  more  closely  represent  the  expected  or  known  shipboard  conditions, 
and  (2)  which  appear  adequate  in  their  present  form.  Those  which  appear  adequate  and 
those  which  could  be  modified  would  necessarily  be  applicable  to  all  equipments  to  as- 
sure satisfactory  operation  aboard  ship.  The  environmental  tests  and  analyses  describing 
shipboard  conditions  occurring  very  rarely  or  never  having  occurred  to  date  to  an  opera- 
tional Navy  ship  are  nuclear  air  blast  analysis  and  shock  tests  so  designed  as  to  simulate 
shock  created  by  a nearby  underwater  blast.  This  shock  test  is  specified  in  MIL-S-901. 

Both  could  be  eliminated  from  MIL-E-16400  as  requirements  for  noncritical  equipments. 

It  has  also  been  noted  that  acceptable  performance  under  vibration  testing  has  been  in- 
dicative of  satisfactory  shock  performance. 

One  type  of  shock  occurring  aboard  ships  that  is  not  addressed  in  MIL-E-16400 
is  shock  created  by  the  muzzle  blast  from  a ship’s  own  guns  impinging  on  external  bulk- 
heads. The  severity  of  this  type  shock  is  dependent  upon  the  distance  lrom  the  gun 
muzzle  to  a bulkhead  and  the  distance  from  this  bulkhead  at  which  equipment  is  mounted. 
In  the  past,  attempts  have  been  made  to  analytically  qualify  equipment  to  this  type 
shock  with  little  or  no  success.  At  times  individual  equipment  specifications  will  require 
installation  aboard  ships  in  a specified  area  with  equipment  performance  being  monitored 
during  gunfire  exercises. 

Certain  tests  and  analyses  describe  conditions  which  can  be  expected  to  occur 
frequently  aboard  Navy  ships  and  should  be  applied  to  all  equipments.  These  are  wind 
speed  analysis  and  icing,  noise,  and  enclosure  tests.  Airborne  and  structureborne  noise 
and  enclosure  tests  should  be  applied  to  all  sheltered  equipments.  These  tests  and  anal- 
yses are  appropriate  in  their  present  form  and  should  be  applied  to  all  equipments  as 
specified. 

MIL-E-16400  requires  all  equipments  to  be  inclination  tested.  This  test  should 
be  required  only  for  equipments  deemed  to  be  position  sensitive  (equipments  whose 
proper  operation  is  dependent  upon  fluid  levels,  position-sensitive  switches,  etc),  since 
this  test  is  a slow  rocking  motion  to  which  most  equipments  are  insensitive. 

Vibration,  temperature,  and  relative  humidity  are  conditions  which  exist  aboard 
ship.  MIL-E-16400  requires  all  equipments  to  be  vibration  tested  as  specified  in  Mil 
STD-167.  Temperature  and  relative  humidity  tesls  arc  specified  in  MI  L-E.- 16400.  Since 


these  conditions  constantly  exist,  it  is  mandatory  that  all  equipments,  both  critical  and 
noncritical,  be  capable  of  satisfactory  operation  over  an  extended  period  of  time  while 
being  exposed  to  them.  Therefore,  vibrational  envelopes  were  established  for  various 
classes  of  Navy  ships,  and  worldwide  extreme  temperature  and  relative  humidity  condi- 
tions were  identified. 

As  a first  step  toward  establishing  environmental  vibration  envelopes  for  various 
classes  of  Navy  ships,  literature  on  hand  was  reviewed.  This  encompassed  48  annual  vi- 
bration  survey  a'ports  from  the  Boston.  Long  Beach.  Norfolk,  Pearl  Harbor.  Philadelphia. 
Portsmouth.  Puget  Sound,  San  Francisco  Bay,  and  Charleston  Naval  Shipyards  which 
covered  the  period  from  1961  to  1973.  From  these  reports  vibration  data  from  more 
than  160  ships  of  the  1)1).  DDG,  DLG,  l)L.  I)F.  DIG.  CVS.  CVA.  ( VAN.  I Pll.  1ST. 
MSO.  PC.  and  SSN  types  were  selected.  NLLC  reports  1577  and  1701  which  are  ac- 
cumulations of  vibration  data  measured  by  NHLC  personnel  aboard  various  Navy  ships, 
and  Naval  Ship  Research  and  Development  Center  (NSRI)C)  Reports  2338  and  233SA 
were  also  reviewed.  Only  vibration  data  for  spaces  containing  or  in  close  proximity  to 
electronic  equipment  were  selected,  and  these  electronic  equipment  spaces  were  in  the 
ship’s  superstructure.  Since  literature  surveys  show  the  most  severe  vibration  conditions 
occur  in  a ship’s  stern,  the  vibration  data  selected  do  not  represent  the  worst-case  con- 
ditions occurring  aboard  ship.  This  survey  also  indicates  that  the  shock  and  vibration 
environment  existing  on  a ship’s  mast  is  minimal,  and  these  data  were  not  used 

The  vibration  data  were  sorted  by  ship  type  and  converted  to  displacement  (inches 
single  amplitude)  vs  frequency  (II/).  A plot  of  these  data  was  made  for  each  class  of 
ship  (fig  A-l-A-8)  by  plotting  data  points  on  multicoordinate  vibration  graph  paper  and 
connecting  peak  values  with  straight  lines.  The  region  under  the  plotted  curve  was  then 
considered  to  be  the  vibration  envelope:  that  is,  any  point  within  the  curve  may  be  a vi- 
bration condition  expected  aboard  that  type  of  ship;  Referring  to  figures  A-l  through 
A-S.  the  frequencies  range  from  I to  45  11/  with  accelerations  seldom  exceeding  0.2  g 
and  displacements  seldom  exceeding  0.030  inch  single  amplitude. 

See  figures  A -9  and  A-10  for  TILC'AM  vibration  test  amplitudes,  figure  A-l  1 
for  a MIL-STD-167  vibration  test  amplitude.  The  THLCAM  vibration  test  amplitudes 
are  listed  in  table  A-l  below. 

TABU  A-l.  INPUT  AM  PI  ITUDLS 

Use  A or  B.  whichever  is  bet t or  suited  lo  the  vibration  test  machine  in  use. 


A (Constant  Displacement)  (see  tig  A ' ) ) 

Frequency  Range  (II/.)  Single  Amplitude  (Inch)  g ~ A (• 

4-10  .03  + .003  .05  - .3 

10-17  .01  ± .001  .10-  3 

17  -30  .003  t .0003  ()‘>  - .28 

30  - 60  .0014.0001  On  - .38 

B (Constant  Acceleration)  (see  tig  A - 1 0 ) 

Frequency  Range  (H/)  Single  Amplitude  ( Inch)  g ATI 

.03  * .003  05  - 2 

03  - .0005  2 * 02 
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I'he  following  vibration  tests  were  developed  from  the  vibration  envelope  informa- 
tion: 

SCOPF.  All  eonimunieations  equipment  intended  for  shipboard  use  under  the 
THLCAM  concept  shall  be  proved  acceptable  under  the  vibration  tests  described 
herein.  Magnitudes  of  input  motions  and  the  associated  frequencies  are  chosen  to 
represent  usual  conditions  existing  aboard  ship  and  do  not  cover  the  vibration  en- 
vironment that  may  exist  when  a ship  has  sustained  battle  damage  and  must  con- 
tinue to  operate.  Users  of  this  test  are  therefore  encouraged  to  be  critical  in 
judging  acceptability  of  the  equipment  under  test. 

BASIS  OF  ACCHPTABI L1TY.  Acceptability  shall  be  judged  as  under  Military 
Standard  M1L-STD-1 67B  (SHIPS).  Mechanical  Vibrations  of  Shipboard  I quip- 
men  t: 

Basis  of  acceptability.  Acceptability  will  be  contingent  upon  the  ability  of 
the  equipment  to  perform  its  function  during  and  after  the  tests  specified 
in  5.1.3.  Minor  damage  or  distortion  will  be  permitted  during  test  providing 
such  damage  or  distortion  does  not  in  any  way  impair  the  ability  of  the  equip- 
ment to  perform  its  principal  functions.  Because  of  the  numerous  types  of 
equipment  covered  by  this  standard,  a definite  demarcation  between  major 
and  minor  failures  cannot  be  specified.  Therefore,  such  decisions  must  nec- 
essarily be  left  to  the  judgment  of  the  test  engineer.  In  general,  a major  failure 
is  one  which  would  cause  maloperation  or  malfunction  of  the  item  of  equip 
merit  for  a long  period.  Nonrepetitive  failures  ot  such  parts  as  vacuum  tubes, 
condensers,  and  wiring,  which  can  be  easily  replaced  or  repaired,  are  generally 
considered  minor  failures.  As  such,  the  repair  could  be  made  and  the  test 
continued  with  no  penalty  to  the  remainder  of  the  equipment.  Sometimes 
the  critical  use  of  the  equipment  will  determine  the  category  of  failure:  that 
is.  a failure  of  a part  in  a lighting  circuit  mav  be  considered  minor.  I lie  same 
failure  in  a reactor  control  circuit  may  be  major.  Thus  the  test  engineer  or 
command  or  agency  concerned,  shall  be  responsible  for  specifying  a major 
or  minor  failure. 

In  addition,  large  amplifications  of  motion  created  by  resonant  response  of  the 
equipment  under  test  arc  to  be  considered  unacceptable  unless,  in  the  opinion  of 
the  vibration  test  engineer,  such  responses  do  not  predicate  early  equipment  failure. 

INTFNT  OF  TFSTS.  Vibration  tests  specified  herein  are  intended  to  locate  reson- 
ances of  the  equipment  under  test,  and  to  impose  a 2 hour  endurance  test  at  those 
frequencies  and  amplitudes. 

TFSTINO  MACII1NFS.  Any  vibration  testing  machine  capable  of  providing  sinus- 
oidal motion,  of  the  frequency  and  amplitudes  specified,  to  the  points  of  attach- 
ment of  the  equipment  under  test  may  be  used.  Means  shall  be  provided  for  con- 
trolling the  direction  of  vibration  of  the  testing  machine  and  for  adjusting  and  mea- 
suring its  frequencies  and  amplitude  of  vibration  to  keep  them  within  prescribed 
limits.  It  the  lower  frequency  limit  of  4 11/  specified  cannot  be  reached,  the  avail 
able  machine  may  be  used  upon  approval  of  the  command  or  agency  concerned 
provided  the  natural  frequencies  of  the  equipment  in  translational  and  rocking 
modes  of  vibration  do  not  lie  below  the  lowest  frequency  ot  the  available  testing 
machine  Th is  may  sometimes  be  determined  by  properly  controlled  transient 
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excitation,  such  as  humping  the  equipment  to  see  whether  low-frequency  reson- 
ances exist.  In  no  case  shall  a vibration  testing  machine  he  used  which  has  a mini- 
mum frequency  greater  than  10  11/ 

METHODS  OF-'  ATTACHMENT,  for  all  vibration  tests,  the  equipment  shall  be 
secured  to  the  vibration  machine  in  the  same  manner  in  which  it  will  be  secured  on 
shipboard.  If  alternative  mountings  of  the  equipment  are  possible  and  contemplated, 
a complete  fl I.CAM  test  shall  be  conducted  in  each  alternative  mounting  configura- 
tion. 

FIXTURES.  When  fixtures  must  be  used  in  attaching  equipment  to  the  vibration 
machine  (for  instance,  when  the  equipment  requires  bulkhead  or  overhead  support), 
the  fixtures  used  shall  be  sufficiently  rigid  to  ensure  that  motion  at  the  points  of 
attachment  of  the  equipment  is  essentially  the  same  as  motion  of  the  main  machine 
platform. 

PORTABLE  EQUIPMENT.  Equipment  which  is  not  intended  tor  permanent  attach- 
ment on  shipboard  shall  be  attached  to  the  vibration  testing  machine  in  a manner 
representative  of  that  in  which  it  will  be  stored  on  board  ship.  In  many  cases,  this 
will  require  that  the  equipment  be  restrained  on  the  vibration  testing  machine  In 
means  of  suitable  small  woven  straps  or  line. 

ORIENTATION  FOR  VIBRATION  Tl  ST.  Equipment  shall  be  installed  on  vibra- 
tion testing  machines  in  such  manner  that  the  direction  of  vibration  will  be  in  turn 
along  each  of  the  three  rectilinear  orientation  axes  of  the  equipment  as  installed  on 
shipboard  vertical,  athwartship.  and  tore  and  alt.  On  a horizontal  vibration  testing 
machine,  the  equipment  may  be  turned  00  degrees  in  the  horizontal  plane  in  order 
to  vibrate  it  in  each  of  the  two  horizontal  orientations.  At  no  time  shall  the  equip- 
ment be  installed  in  any  other  way  than  its  normal  position.  Because  no  particular 
orientation  of  portable  equipment  can  be  considered  normal,  orientation  of  portable 
equipment  for  vibration  testing  shall  be  left  to  the  discretion  of  test  engineer. 

RESILIENT  MOUNTINGS.  Equipment  which  is  to  be  installed  on  resilient  mounts 
on  board  ship  shall  be  installed  on  those  mounts  for  vibration  testing.  Resilient 
mountings  integral  to  the  equipment  (intended  to  protect  sensitive  deuces  within 
the  equipment)  shall  be  left  in  place. 

VIBRATION  TESTS.  Each  of  the  tests  specified  herein  shall  be  conducted  separ- 
ately in  each  of  three  principal  axes  of  the  equipment  under  test.  Each  type  ol 
test  (exploratory  vibration,  variable  frequency,  and  endurance,  below)  shall  be 
completed  in  all  three  axes  before  proceeding  to  the  next  type  of  vibration  test 
During  the  tests,  the  equipment  shall  be  secured  to  the  test  machine  and  oriented  as 
specified  in  METHODS  OF  ATTACHMENT  and  EIXTURI  S.  above.  I he  equip- 
ment shall  be  energized  and  performing  its  normal  functions.  Appropriate  test 
equipment  shall  be  applied  to  the  equipment  under  test  to  reveal  vibration-caused 
electrical  malfunctions  which  are  not  manifested  in  mechanical  responses.  Unless 
otherwise  directed,  the  test  shall  be  discontinued  if  major  failure  occurs,  and  the 
entire  vibration  test  series  repeated  following  repair  or  correction  of  deficiencies. 

An  entirely  new  test  specimen  may  be  substituted  for  the  retest,  but  the  substitu- 
tion must  be  noted  in  the  test  report  furnished.  (See  TEST  REPORT,  below  t 

I XPLORATORY  VIBRATION  I I SI  I'o  determine  the  presence  ol  possible 
harmful  resonances  in  the  equipment  under  lest,  vibration  in  the  frequence  range 
from  4 Hz  (or  the  lowest  attainable  frequence  not  to  exceed  10  11/)  to  (■>()  Hz  shall 
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be  applied.  Amplitude  of  input  vibration  shall  be  only  great  enough  to  determine 
resonant  responses  of  the  equipment  under  test  and  to  obtain  an  approximation  of 
the  amplification  factor.  In  no  ease  shall  input  amplitudes  exceed  those  specified 
for  the  VARIABLE  FREQUENCY  TFST.  below.  The  frequency  range  is  to  be  swept 
from  low  to  high  frequency  at  a rate  not  greater  than  4 11/  per  minute  nor  less  than 

1 II/  per  minute.  Resonant  frequencies  and  approximate  amplification  factors 
shall  be  noted.  If  believed  necessary  by  the  test  engineer,  resonances  noted  during 
the  scan  shall  be  reestablished,  peaked,  and  held  briefly  to  permit  adequate  docu- 
mentation. Resonances,  how  they  were  manifest,  their  frequencies  (±0.5  Hz),  and 
their  approximate  amplification  factors  shall  be  included  in  the  test  report.  (See 
TEST  REPORT,  below.) 

VARIABLE  FREQUENCY  TEST.  The  equipment  under  test  shall  be  subjected  to 
vibration  between  4 11/  (or  the  lowest  attainable  frequency  not  to  exceed  10  ll/i 
and  60  Hz  with  input  amplitudes  as  shown  in  table  A-l . Vibration  input  shall  be 
at  discrete  integral  frequencies  (±0.5  Hz),  and  each  frequency  shall  be  maintained  for 
a period  of  5 minutes.  Peak  responses  of  the  equipment  under  test  are  likely  to  occur 
at  nonintegral  frequencies.  Mechanical  nonlinearity  of  the  equipment  under  test 
will  cause  a shift  in  the  frequency  of  maximum  response  as  documented  in  EXPLORA- 
TORY VIBRATION  TEST,  above.  Therefore,  as  frequency  changes  are  made,  the 
test  engineer  shall  carefully  observe  the  equipment  under  test  and  peak  any  response, 
even  though  it  may  occur  at  a nonintegral  frequency,  hold  the  response  long  enough 
to  obtain  documentation,  and  then  move  on  to  the  next  integral  frequency. 

ENDURANCE  I 1ST.  The  equipment  under  test  shall  be  vibrated  tor  a period  of 

2 hours  at  each  of  the  resonances  noted  m VARIABl  I I Rl  QU1  NO  1 1ST.  with 
input  amplitudes  as  specified  in  table  A-l  In  the  event  no  resonances  have  been 
noted,  the  equipment  shall  he  vibrated  for  2 hours  at  ot)  II/. 

TEST  RE  POR  I The  test  report  to  be  furnished  the  command  or  agency  concerned 
In  the  testing  laboratoix  shall  include  detailed  descriptions  ol  an\  damageorm.il 
functioning  incurred  and  at  what  stage  in  the  tests  it  occurred.  When  possible, 
photographs  of  physical  damage  shall  be  included.  Recommendations  are  desired 
as  to  what  corrective  measure,  it  am  . should  be  taken.  It  shall  also  include  other 
pertinent  information,  such  as  the  overall  dimensions  ol  the  equipment,  its  weight, 
approximate  location  ol  the  center  ot  gra\ it \ . a sketch  or  photograph  ot  the 
methods  used  in  mounting  it  on  the  test  machines,  a listing  ol  resonances,  frequen- 
cies at  which  resonances  occurred,  and  t heir  approximate  amplilicat ion  factors 

A temperature/relative  humidity  profile  was  developed  for  1 1 I C AM  much  like  the 
II  1 CAM  vibration  profile.  Data  that  had  previously  been  collected  from  instrumented 
shipboard  surveys  and  weather  reports  had  been  plotted  as  temperature  vs  relative  huumlin 
and  were  compared  with  MIL-1- 1 6400  requirements  as  shown  in  figure  A-l  2.  I he  bounds  of 
the  natural  occurrences  plotted  became  the  El  I CAM  limits.  Referring  to  figure  A- 15.  an 
easily  applied  test  can  be  described  by  a temperature/relative  humidity  trapezoid  with  cor- 
ner points  at  68  degrees  E with  05-percent  relative  humidity.  '>5  degrees  I with  05-percent 
relative  humidity.  I 22  degrees  with  65-percent  relative  humidity,  and  68  degrees  I with 
65-percent  relative  humidity.  This  test  is  conducted  by  making  five  complete  cycles  around 
the  trapezoid  starting  and  finishing  at  05  degrees  E and  05-percent  relative  humidity  I aeh 
test  condition  is  maintained  for  5 hours  with  I hour  allowed  for  the  transition  between 
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points,  thereby  making  each  cycle  24  hours  long.  This  test  differs  from  MIL  -1  1 <>400  re- 
quirements in  that  it  does  not  require  the  simultaneous  high-teinperature/high-relative- 
humidity  condition  of  122  degrees  F with  95-percent  relative  humidity.  This  condition 
does  not  exist  naturally.  The  lower  temperatures  and.  transitions  simulate  conditions 
existing  aboard  ship  when  air  conditioning  equipment  fails  and  a communications 
compartment  must  be  opened  to  ambient  temperature  and  relative  humidity  levels. 

Both  the  TFLC AM-developed  vibration  and  temperature/relative  humidity  tests 
are  not  as  severe  as  their  parent  MIL  tests.  This  condition  resulted  because  the  1TL- 
CAM  tests  were  not  developed  to  test  equipment  for  survivability  under  abnormal  ship- 
board environments,  but  to  provide  confidence  the  equipment  will  endure  the  usual  en- 
vironments. All  shipboard  mounted  equipments  are  constantly  exposed  to  the  vibration 
and  temperature/relative  humidity  environments  during  at-sea  operation;  therefore,  testing 
to  and  determining  equipment  survivability  under  these  conditions  should  be  considered 
as  minimum  requirements.  I quipment  should  be  tested  to  the  inclination,  electrical 
transient,  noise,  and  enclosure  tests,  as  applicable,  specified  in  MIL-1- 1 6400  to  increase 
the  confidence  level  of  shipboard  survivability. 

To  date,  four  commercial  equipments  have  been  chosen  for  T1  LCAM  vibration 
and/or  temperature/relative  humidity  testing.  These  were  a floppy  disk  memory  unit,  a 
digital  plotter,  a cassette  tape  recorder,  and  a single  sideband  radiotelephone.  The  radio- 
telephone has  not  been  completely  tested  due  to  initial  problems  requiring  correction 
by  the  manufacturer.  All  the  equipments  selected  for  testing  under  the  Tl  LCAM  pro- 
gram are  assumed  to  be  readily  available  so  that  redundant  equipment  can  be  on  hand 
All  tests  that  will  be  applied  to  these  equipments  represent  noncombatant  situations. 

The  floppy  disk  memory  unit  was  selected  for  testing  because  it  is  an  ideal  can- 
didate for  use  with  shipboard  minicomputers.  To  monitor  test  performance,  alphabetical 
characters  from  A through  P were  stored  on  the  inside  track  of  the  disk.  The  characters 
were  transferred  from  the  inside  track  to  the  outside  track,  printed  out  via  teletype,  and 
then  compared  with  the  inside  track.  This  was  done  continuously  during  vibration  test- 
ing and  at  the  end  of  each  temperature/relative  humidity  24-hour  cycle.  When  a discre- 
pancy occurred,  the  teletype  printed  out  DISK  NOT  RI  ADY. 

The  floppy  disk  memory  unit  was  first  subjected  to  the  Tl  LCAM  vibration  test. 
The  constant  displacement  method  was  used.  The  only  vibration-related  discrepancy 
noted  during  the  test  was  a resonance  in  the  circuit  board  occurring  at  51  11/  This 
condition  was  eliminated  by  attaching  the  circuit  board  corners  to  the  chassis.  1 wo 
DISK  NOT  RFADY  messages  were  printed  out  by  the  teletype  during  the  lest  but 
they  were  not  repeatable,  and  no  vibration-related  reason  could  be  found. 

The  Tl  LCAM  temperature/relative  humidity  test  was  conducted.  The  test  was 
started  at  95  degrees  F and  b5-percent  relative  humidity.  As  stated,  the  alphabetical 
characters  were  transferred  from  the  inside  track  to  the  outside  track,  printed  out  In 
the  teletype,  and  compared  with  the  inside  track  at  the  end  of  each  24-hour  cycle.  I he 
unit  was  turned  off  during  the  24-hour  cycle.  When  the  unit  was  turned  on.  the  disk 
required  approximately  50  seconds  to  warm  up  before  it  would  respond  to  the  teletype 
control.  Moisture  collecting  inside  the  unit  that  had  to  be  exhausted  In  the  unit  s in- 
ternal cooling  fan  before  proper  operation  could  be  obtained  was  considered  the  cause 
for  this  50-second  delay. 

To  increase  the  confidence  level  that  the  (loppy  disk  memory  unit  would  stir 
vive  aboard  ship,  the  supply  line  voltage  and  frequency  tests  described  in  Mil  l - I (>4001 
section  4.5.4  were  performed.  No  abnormal  effects  were  apparent  during  the  steads  - 
state  and  transient  tests.  The  teletype  did  print  out  the  DISK  NOT  Kl  ADY  message 
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during  power  dropout  tests.  This  condition  was  deemed  insignificant  as  it  occurred  only 
when  the  power  dropped  out  exactly  as  its  waveform  crossed  the  neutral  axis,  and  this 
would  rarely  occur  naturally.  This  condition  had  no  detrimental  effect  on  the  unit,  which 
operated  properly  when  the  power  returned  to  normal. 

These  tests  performed  on  the  floppy  disk  memory  unit  indicate  that  it  should 
deliver  satisfactory  performance  if  installed  aboard  Navy  ships. 

The  digital  plotter  was  submitted  for  vibration  testing  by  the  Naval  Undersea 
Center  (NUC)  The  plan  was  to  perform  an  exploratory  vibration  test  (per  MIL-STO-lfi?) 
only  in  each  of  the  three  mutually  perpendicular  axes  to  identify  resonant  frequencies. 

During  actual  testing,  input  levels  had  to  be  reduced  when  resonances  were  reached 
to  avoid  damaging  the  plotter.  These  reduced  levels  were  compared  with  and  were  equal 
to  or  slightly  larger  than  the  TILCAM  vibration  levels.  Throughout  this  exploratory 
\ihration  test  the  plotter  operated  properly,  and  at  no  time  did  the  trace  become  discon- 
tinuous or  rippled.  The  tracing  capability  of  the  plotter  was  of  prime  importance  during 
the  test.  The  plotter  can  be  considered  acceptable  insofar  as  vibration  is  concerned,  but 
before  any  conclusions  can  be  reached  as  to  the  overall  suitability  of  this  plotter  for 
shipboard  use.  further  testing  and  analysis  will  be  required. 

The  cassette  tape  recorder,  which  is  being  used  in  a shore-based  Navy  receiver, 
was  subjected  to  the  TI  LCAM  temperature/relative  humidity  and  vibration  tests.  Per- 
formance was  measured  during  testing  by  making  voice  recordings  and  by  operating 
playback,  fast  forward,  and  fast  reverse  controls. 

The  Tl  LCAM  temperature/rdative  humidity  test  was  run  for  four  complete 
cycles  continuous  hours).  Recorder  performance  was  monitored  just  as  the  95  de- 
gree F and  05-percent  relative  humidity  condition  was  reached.  The  recorder  was 
turned  off  during  the  rest  of  the  cycle.  The  only  abnormality  noted  was  slight  sluggish- 
ness in  the  reverse,  or  rewind,  mode  of  operation.  This  sluggishness  was  not  deemed  de- 
trimental to  recorder  performance,  and  it  disappeared  when  the  recorder  was  returned 
to  room  temperature 

The  constant  displacement  with  constant  level  crossover  TI  LCAM  vibration  test 
(see  fig  A- 1 0 ) was  used  to  test  the  recorder  in  the  vertical  axis.  During  the  complete  i 

exploratory,  variable  frequency,  and  endurance  tests  no  resonances,  performance  degrad- 
ation. or  other  discrepancies  were  noted. 

Since  the  recorder  performed  satisfactorily  during  TI  LCAM  vibration  testing 
in  the  vertical  axis,  the  more  severe  requirements  of  MIL-STD-1  f>7  were  applied  in  the 
remaining  vibration  test.  Temperature  relative  humidity  and  vibration  test  results  in- 
dicate this  recorder  should  operate  satisfactorily  aboard  ship,  even  under  abnormal 
vibration  conditions. 

The  literature  survey  of  shipboard  environmental  conditions  is  continuing.  Other 
equipments  are  being  investigated  as  possible  candidates  to  be  exposed  to  TIT  CAM- 
developed  tests,  and.  where  no  environmental  data  exist,  the  possibility  of  performing 
at-sea  measurements  is  being  considered. 

New  ship  classes  such  as  the  Newport  class  of  LST.  LCC,  1000  series  DF.  etc. 
should  be  surveyed  and  underway  measurements  made  to  document  their  environments. 

(See  Jane's  Fighting  Ships.)  Fquipment  packaging  and  transportation  environments 
have  not  been  investigated.  Many  instances  have  occurred  in  which  equipment  has  been 
damaged  during  transportation  due  to  inadequate  packaging  and  rough  handling  'Time 
and  effort  should  be  expended  to  eliminate  some  of  the  inadequacies  in  these  areas. 
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DISCUSSION 

A number  of  government  studies  have  sought  answers  to  the  problem  of  reducing 
the  costs  of  acquiring  and  using  military  items.1 * * 4 *’6  It  was  originally  intended  here  to  pre- 
sent a matrix  of  recommendations  vs  each  study  to  provide  an  overview  of  ways  and  means 
for  cost  reduction.  However,  the  most  recent  study  to  date  with  a published  report  has 
incorporated  the  results  of  the  other  studies  with  those  of  its  own,  and  thus  represents 
current  and  official  thinking  across  the  board  on  cutting  electronic  system/equipment  costs. 

It  is  the  “Electronics-X”  study.  In  addition,  Electronics-X  is  especially  pertinent  and  sig- 
nificant to  the  TELCAM  project  since  it  addresses  itself  specifically  to  electronic  systems 
and  equipment,  whereas  the  other  studies  either  are  of  a very  general,  statute-oriented 
nature  (eg,  the  “Report  of  the  Commission  on  Government  Procurement”)  or  focus  on 
some  one  aspect  of  reducing  costs  (eg,  the  design-to-cost  emphasis  of  the  “Report  of  the 
Defense  Science  Board  Task  Force  on  Reducing  Costs  of  Defense  System  Acquisitions”). 

Therefore,  rather  than  a matrix,  a tabulation  of  Electronics-X  recommendations  is  pre- 
sented. In  the  few  instances  where  other-study  recommendations  pertinent  to  TELCAM 
are  not  included  among  Electronics-X  recommendations,  they  are  additionally  listed  and 
specially  noted.  This  tabulation  may  be  considered  all-inclusive  with  respect  to  current 
government-study  recommendations  for  reducing  the  costs  of  acquiring  and  using  elec- 
tronic systems  and  equipment. 


RECOMMENDATIONS 

COST-DATA  COLLECTION  AND  REPORTING  SYSTEMS 

1.  ***  A systematic  effort  should  be  undertaken  to  develop  a step-wise  imple- 

mentation of  a complete  and  uniform  cost  accounting  system  throughout 
DoD.  with  emphasis  on  valid  input  data.  As  a first  step,  a marginal  cost  sys- 
tem using  sampling  techniques  for  support-cost  inputs  should  be  implemented. 
The  system  must  then  evolve  to  cover  full  costs  of  both  acquisition  and 
support. 


1 Report  R-195,  Institute  for  Defense  Analyses,  Science  and  Technology  Division.  Electronics-X:  A Study 
ot  Military  Electronics  With  Particular  Reference  to  Cost  and  Reliability  ■ January  1974.  prepared  for 
Defense  Advanced  Research  Projects  Agency,  DDR&E 
* Report  of  the  Commission  on  Government  Procurement . December  1972,  to  the  Congress  of  the  United 
States 

1 Report  ot  the  Detense  Science  Board  Task  force  on  Weapon-System  Simplification.  Summer  ot  1970, 

DDR&E 

4 Report  ot  t lie  Detense  Science  Board  I ok  force  on  Avionics.  February  1973,  DDR&E 

h Report  ot  the  Dctensc  Science  Board  1 ask  force  on  Rcducinti  Costs  ot  Defense  System  Acquisitions, 
March  15.  1973.  DDR&E 

6Cost  Growth  in  Major  Weapons  S\  stems.  March  2(>.  1973,  report  by  the  Comptroller  General  of  the  United 
States  to  the  Committee  on  Armed  Services.  House  of  Representatives 
***Highcst  priority. 
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2.  **  A central  organization  within  OSD  should  be  designated  to  organize  this 
cost  information  system  and  to  coordinate  the  efforts  of  responsible  service 
elements. 

3.  To  test  and  exercise  the  system,  each  service  major  procurement  command 
should  designate  certain  electronic  systems  for  review  of  cost  reporting  require- 
ments. Appropriate  steps  should  be  taken  to  ensure  consistency  among  the 
report  outputs,  complete  record  retrieval,  and  periodic  validation  of  the  reported 
costs.  These  records  should  be  centrally  located  and  should  be  made  available 
to  the  cost  analysis  community. 


RELIABILITY  DATA  COLLECTION  AND  REPORTING  SYSTEMS 

4.  ***  In  each  major  producer  command  (AMC.  NMC.  AFSC.  AFLO.  establish 
(or  broaden)  a system  for  competent  technical  reporting  of  reliability,  avail- 
ability, and  maintainability  (RAM)  and  marginal  cost  feedback  information 
from  the  field  on  selected  systems  and  equipments,  using  sampling  methods. 

5.  **  Organize  a RAM  Data  Systems  Task  Force,  representing  the  several  ser- 
vices and  chaired  by  OSD,  to  study  and  compare  the  relative  cost-effectiveness 
of  a routine  maintenance  data  collection  system  (such  as  3M  or  AFM  66-1) 
with  a sampled-data  collection  system  (such  as  TAMMS-SDC). 

6.  Establish  a new  RAM  Information  Exchange  Program  at  the  electronic  equip- 
ment level  in  a form  patterned  after  the  Government-Industry  Data  Exchange 
Program  (GIDEP). 

REQUIREMENTS  AND  ACQUISITION  DECISIONS 

*7.  ***  In  exploring  and  establishing  a system  requirement,  give  performance, 

physical  characteristics,  cost,  quantity,  and  schedule  equal  status  from  the 
beginning,  and  perform  tradeoffs  among  these  early  in  the  game. 

8.  ***  In  major  system  developments,  separate  vehicle  and  electronic  subsys- 
tem initial  operational  capabilities,  where  possible,  and  develop  the  electronics 
independently.  Consolidate  like  subsystem  or  equipment  requirements  wher- 
ever feasible. 

9.  ***  Increase  visibility  to  top-level  management  of  potentially  cost-driving 
developments  of  electronic  subsystems  associated  with  major  systems,  by 
instituting  suitable  review  prior  to  each  DSARC  review.  As  appropriate,  pro- 
vide for  a similar  visibility  to  managers  of  developments  of  less-than-major 
electronic  subsystems  and  equipments  by  refocusing  reviews  to  make  them 
analogous  to  DSARC  reviews,  but  at  lower  management  levels. 

*10.  ***  Give  increased  consideration  to  product  improvement  programs  as  a 

means  of  fulfilling  new  requirements,  as  opposed  to  instituting  wholly  new 
development  programs. 

•Most  directK  applicable  to  project  manager/engineer  day-to-da\  undertakings. 

**lligh  priori t v 
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*11.  ***  Select  technology  and  performance  objectives  for  new  developments  con- 

servatively (ie,  low  on  the  cost-performance  curve),  except  in  cases  where  mili- 
tary necessity  imposes  an  overriding  need  for  risk-taking  to  achieve  extremes  of 
performance.  Allow  for  uncertainty  in  establishing  the  corresponding  system 
requirements. 

*12.  ***  Iterate  requirements  and  acquisition  decisions  if  performance,  character- 

istics, cost,  quantity,  or  schedule  departs  significantly  from  initial  plans  during 
development.  Establish  criteria  to  trigger  such  iteration. 

13.  (From  ref  2)  For  major  systems,  start  new-system  acquisition  programs  with 
agency-head  statements  of  needs  and  goals  that  have  been  reconciled  with 
overall  agency  capabilities  and  resources.  State  program  needs  and  goals  in- 
dependent of  any  system  product.  Assign  responsibility  for  responding  to 
statements  of  needs  and  goals  to  a single  agency  component,  when  the  mission 
need  is  clearly  the  responsibility  of  that  one  component,  or  to  two  or  more 
agency  components  when  competition  is  formally  recognized,  with  each 
offering  alternative  system  solutions  when  the  mission  responsibilities  overlap. 

14.  (From  ref  2)  Support  the  general  fields  of  knowledge  that  are  related  to  an 
agency’s  assigned  responsibilities  by  funding  private-sector  sources  and  govern- 
ment in-house  technical  centers  to  do  basic  and  applied  research,  proof-of- 
concept  work,  and  exploratory  subsystem  development.  Restrict  subsystem 
development  to  less  than  fully  designed  hardware  until  identified  as  pari  of 

a system  candidate  to  meet  a specific  operational  need. 

15.  (From  ref  2)  For  major  systems,  create  alternative  system  candidates  by  (a) 
soliciting  industry  proposals  for  new  systems,  with  each  contractor  free  to 
propose  system  technical  approach,  subsystems,  and  main  design  features  on 
the  basis  of  a statement  of  government  mission,  time,  cost,  and  operational 
requirements;  and  (b)  sponsoring  the  most  promising  system  candidates  as 
selected  by  agency  component  heads,  using  a team  of  experts  from  inside  and 
outside  the  agency  component  development  organization  as  a review  paiv'l. 

DESIGN  TO  COST 

*l(>.  ***  Choose  easily  defined  design-to-cost  (DTC)  targets  such  as  unit  produc- 

tion or  flyaway  costs  (rather  than,  for  example,  the  presently  still  ill-defined 
life-cycle  costs;  but  see  following  subparagraph  to  this  recommendation). 
Establish  such  targets  early,  (lennilting  successive  revisions  during  development, 
contractual  commitment  to  a unit  cost  for  low.-rate  initial  production  at  the 
start  of  such  production,  and  another  contractual  commitment  for  unit  cost 
at  the  start  of  full-scale  production  for  systems  to  be  procured  in  quantity. 
Flexibility  to  revise  cost  targets  should  decrease,  and  these  should  be  based 
increasingly  on  firm  experience  as  the  development-to-production  cycle 
progresses. 

If  the  equipment  is  to  be  maintained  by  the  supplier  under  long-term  warranty, 
the  design-to-cost  target  can  be  established  as  the  sum  of  the  production  cost  and  total 
warranty  cost;  this  sum  may  be  considered  a surrogate  tor  life-cycle  cost.  But  it  military 
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maintenance  is  contemplated,  establishing  life-cycle  cost  as  a design-to-cost  target  is  not 
now  appropriate  because  of  the  inadequacy  of  current  knowledge  of  the  cost  to  the 
government  of  military  maintenance,  and  of  the  dependence  of  these  costs  on  equipment 
parameters. 

*17.  ***  Establish  explicit  limits  of  deviation  from  "desired"  performance/ 

characteristics/cost/schedule/quantity  requirements,  and  authorize  program 
managers  to  trade  off  freely  among  these  separate  requirement  parameters 
within  the  established  limits.  Establish  "desired”  parameters  anil  permissible 
deviations  such  that  tradeoffs  are  in  fact  possible  and  not  subject  to  hidden 
constraints  due  to  technical  feasibility,  absolute  force  requirements,  or  avail- 
able budgets. 

18.  ***  To  the  extent  feasible,  maintain  design  and  price  competition  throughout 

the  acquisition  process,  especially  for  components  and  subsystems. 

]i)  ** * in  the  contractor-selection  process,  ensure  that  performance  and  cost  are 

considered  together  rather  than  evaluated  separately. 

*20.  **  Tit  is  stidy  identified  only  one  design-to-cost  acquisition,  namely,  the  Navy 

electronic  warfare  suite,  that  uses  the  approach  of  specifying  equipment  needs 
and  requirements  functionally,  leaving  it  to  the  competing  contractors  to 
propose  optimal  development  and  production  strategies  to  maximize  payot t 
to  both  the  government  and  the  contractors,  and  including  maintenance 
strategies  among  the  variables.  More  experience  with  this  approach  should 
be  acquired. 

21.  Increase  the  number  of  design-to-cost  acquisitions  of  electronic  subsystems 
designated  as  "experimental”  for  observation  and  extraction  of  "lessons  learned." 
In  further  experimental  design-to-cost  acquisitions,  seek  wider  variation  ol  the 
management  variables  relevant  to  design  to  cost  (for  example,  tradeoffs  among 
requirements,  program  manager’s  freedom  to  trade  off,  types  of  contracts). 

The  services  should  publish  "lessons  learned"  periodically  to  maximize  the 
pool  of  explicitly  analyzed  experience  available  to  all. 

22.  **  Review  the  contracting  procedures  associated  with  design-to-cost  contracts, 
modify  those  that  inhibit  requisite  design-to-cost  flexibility , and  incorporate 
the  modifications  in  the  ASI’R,  if  necessary. 


DESIGN  FOR  IMPROVED  RELIABILITY 

*22  ***  Limit  the  complexity  of  new  subsystem  or  equipment  designs  (as  mea- 

sured by  criteria  such  as  unit  production  cost  or  parts  count)  to  a level  consis 
tent  with  the  reliability  required  by  a mission  analysis.  Require  evidence  ol 
compliance  as  a preliminary  to  DSARC  review  for  electronic  subsystems  of 
major  systems,  and  as  a preliminary  to  sub-DSARC  review  for  independently 
developed  electronic  subsystems. 

*24.  ***  Require  contractually  the  in-plant  use  of  a formal  management  method- 

ology, such  as  methods  using  Duane-curve  monitoring,  to  ensure  reliability 
growth  in  electronic  equipments  and  systems. 
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*25.  ***  Use  long-term  contractor  maintenance  warranties  to  motivate  the  con- 

tractor to  design  tor  minimum  life-cycle  cost.  (See  later  recommendations 
on  warranties  for  further  details.) 

*26.  ***  Specify  the  reliability  of  electronic  equipments  or  systems  to  he  consis- 

tent with  predictions  based  on  their  anticipated  complexity  (or  unit  produc- 
tion cost,  as  a surrogate  for  complexity). 

27.  ***  Undertake  redesign  of  selected  equipments  witli  the  specific  objective  ot 

improving  reliability  while  holding  performance  constant. 

DESIGN  TO  FACILITATE  COMPETITION 

*28.  ***  Lay  the  groundwork  for  future  design  and  price  competition  through 

production  and  for  ready  replacement  of  old  designs  by  new-generation 
equipment  by  ensuring  the  interchangeability  of  similar  equipments  intended 
for  similar  applications.  Accomplish  this  by  including  mechanical,  electrical, 
and  environmental  interface  standards  for  each  unit  as  part  of  military 
electronic  equipment  specifications. 

Require  design  interchangeability  when  production  competition  or  design  upgrading 
is  foreseen  as  desirable  or  likely.  Equipment  classes  that  are  judged  ripe  for  initial  appli- 
cation of  interface  standardization  are  airborne  communication,  navigation,  identification, 
and  weather  radar;  vehicular  communication;  and  modular  electronics  packages  tor  tactical 
missiles. 

*29.  ***  Modify  approval  processes  for  enginceiing  change  proposals  to  expedite 

incorporation  by  suppliers  of  internal  design  improvements  to  enhance  relia- 
bility and  performance  or  inclusion  of  new  technology  to  meet  competition 
during  the  procurement  cycle  and  even  after  deployment,  it  the  suppliers  ire 
called  upon  to  maintain  their  equipment.  But  keep  rigid  control  over  inter 
face  configurations  to  ensure  interchangeability. 

30.  ***  Obtain  multiple  developments  of  equipments  conforming  to  interface 
specifications.  Where  the  potential  market  for  the  equipment  is  large  enough, 
encourage  industry-financed  development;  otherwise,  procure  multiple  develop 
ments  under  government  contracts. 

31.  ***  Facilitate  government  testing  and  qualification  of  designs  offered  in 
compliance  with  the  specifications,  whether  or  not  the  designs  were  developed 
under  government  contract.  Flan,  prepare,  and  provide  for  retesting  and 
requalification  of  modified  designs  submitted  in  production  competitions 
subsequent  to  the  initial  competition. 

*32.  ***  To  overcome  the  potential  problem  of  spare-parts  stocking  and  field 

repair  of  multiple  equipment  configurations,  make  use  of  depot  repair  or 
supplier  maintenance  under  warranty.  In  the  field,  replace  rather  than  rep.iit 
failed  replaceable  units  of  equipment.  Include  warranty  requirements  when 
initiating  development. 
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**  To  achieve  multiple-source  availability,  rely  on  performance  specifications 
plus  environmental  and  interface  requirements  (ie.  “form.  fit.  function”  spe- 
cifications) to  define  equipment,  rather  than  imposing  detailed  specifications 
on  parts,  processes,  materials,  and  internal  configuration. 

34.  To  broaden  the  markets  for  competitive  suppliers,  encourage  the  evolution 
of  multiservice  interface  standards. 

35.  (Front  ref  2)  Maintain  competition  between  contractors  exploring  alternative 
systems  by  (a)  limiting  commitments  to  each  contractor  to  annual  fixed-level 
awards,  subject  to  annual  review  of  their  technical  progress:  (b)  assigning 
agency  representatives  to  advise  competing  contractors  in  developing  require- 
ments for  each  candidate  system  as  tests  and  tradeoffs  are  made;  and  (cl 
concentrating  the  activities  of  agency  development  organizations,  government 
laboratories,  and  technical  management  staffs  on  monitoring  and  evaluating 
contractor  efforts,  and  on  participating  in  those  tests  critical  to  determining 
whether  the  candidate  system(s)  should  be  continued. 

3b.  (From  ref  2)  Limit  premature  major-system  commitments  and  retain  the 

benefit  of  competition.  Ibis  should  be  accomplished  by  conducting  compe- 
titive demonstrations  of  candidate  systems  after  a determination  by  agency 
beasts  to  do  so  and  on  these  steps:  (a)  choosing  contractors  according  to  their 
relative  technical  progress,  the  remaining  uncertainties,  and  the  economic 
constraints;  (b)  providing  selected  contractors  with  the  operational  test  con- 
ditions, mission  performance  criteria,  and  lifetime  ownership  cost  t actors  to 
be  used  in  the  final  system  evaluation  and  selection:  (c)  proceeding  with 
final  development  and  initial  production,  and  with  commitments  to  a tirm 
date  for  operational  use  (after  agency  needs  and  goals  are  reaffirmed,  com- 
petitive demonstration  results  prove  that  the  chosen  technical  approach  is 
sound,  and  the  definition  of  a procurement  program  is  practicable):  and  id) 
strengthening  each  agency's  cost  estimating  capability. 

37.  (From  ref  2)  Should  an  agency  component  (responsible  developer)  determine 
it  should  concentrate  development  resources  on  a single  system  rather  than 
fund  competitive  system  candidates,  require  agency-head  approval  be  I ore 
proceeding.  When  such  single-system  development  is  to  be  pursued,  la) 
establish  a strong  centralized  program  office  within  the  agency  component  to 
take  direct  technical  and  management  control  of  the  program:  (hi  integrate 
selected  technical  and  management  contributions  from  in-house  groups  and 
contractors:  <c)  select  contractors  with  proved  management,  financial,  and 
technical  capabilities,  as  related  to  the  problem,  and  use  cost  reimbursement 
contracts  for  high-techriical-risk  portions  of  the  program,  and  <dl  estimate 
program  cost  within  a probable  range  until  the  major  system  reaches  the  tinal 
development  phase. 

38.  (From  ref  2 > Withhold  agency -head  approval  and  congressional  commitments 
for  full  production  and  use  of  new  major  systems  until  the  need  has  been 
reconfirmed  and  the  system  performance  lias  been  tested  and  evaluated  in  an 
environment  that  closely  approximates  expected  operational  conditions 
Regarding  T&F.  (a)  continue  efforts  to  strengthen  lest  and  evaluation  cupa 
bilities  in  the  services,  (hi  establish  an  agency  wide  definition  ol  the  scope  oi 
operational  test  and  evaluation,  and  (cl  establish  in  each  agency  component 
an  operational  IfSt  1 activity  separate  from  its  developer  and  user  organizations 
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39.  (From  re.*'  2)  Use  contracting  as  an  important  tool  of  system  acquisition,  not 
as  a substitute  for  management  of  acquisition  programs.  In  so  doing,  (a)  set 
policy  guidelines  within  which  experienced  personnel  may  exercise  judgment 
m selectively  applying  detailed  contracting  regulations,  (lit  develop  simplified 
contractual  arrangements  and  clauses  for  use  in  awarding  final  development 
and  production  contracts  for  demonstrated  systems  tested  under  competitive 
conditions,  and  (cl  allow  contracting  officials  to  use  priced  production  options 
if  critical  test  milestones  have  reduced  risk  to  the  point  that  the  remaining 
development  work  is  relatively  straight-forward. 

40.  (From  ref  2)  Within  each  agency  and  agency  component,  unify  policy-making 
and  monitoring  responsibilities  for  major  system  acquisitions,  including  mini- 
mization of  management  echelons,  staff  reviews,  coordinating  points,  procedures, 
reporting,  and  paperwork  within  both  industry  and  government. 

PRODUCTION 

41.  ***  Where  the  quantity  to  be  bought  is  large  enough,  depart  from  the  con- 
ventional approach  of  aggregating  procurements  into  a single  large  buy 
intended  to  take  advantage  of  "learning  curves."  Instead,  fragment  the 
procurements  into  sequential  buys,  inviting  design  and  price  competition  on 
each  buy  by  the  several  suppliers  of  qualified  interchangeable  equipments 

42.  **  The  government  must  assure  prospective  suppliers  that  there  will  be  future 
design  and  price  competition.  One  method  of  so  doing  is  to  analyze  and 
publish  future  needs  and  a schedule  of  planned  competitive  buys. 

43.  ***  The  government  must  provide  assurance  that  new  or  improved  designs 
will  be  given  full  consideration  in  future  competitions  if  they  meet  the  form, 
tit.  and  function  requirements  that  ensure  interchangeability  with  prior 
designs.  This  implies  the  need  for  inclusion  of  interface  requirements  in  gov 
eminent  specifications. 

44.  ***  The  government  must  offer  to  perform  and  must  be  prepared  to  perform 
laboratory  tests  and  evaluations  of  the  actual  hardware  prototypes  of fereil  by 
bidders  or  prospective  competitors  in  order  to  qualify  the  designs  tor  current 
and  future  competition. 

45  When  it  is  desirable  and  necessary  to  sustain  competition,  award  tractions  ot 
each  buy  to  two  or  three  competitors  in  proportion  to  the  merit  of  their 
respective  designs  and  prices,  rather  than  making  the  award  on  a winner  take 
all  basis 


RKPROCURFMFNT 

4b  **  In  selected  development  contracts  where  subsequent  competitive  leprocurc 
nient  is  anticipated,  the  government  should  provide  a payment  to  the  dcvelopci 
for  each  accepted  unit  produced  under  government  contract  from  the  dcvcl 
oper's  design  by  a supplier  other  than  the  developer  I his  payment  should 
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constitute  a deferred  part  of  the  compensation  for  the  reprocurement  data 
package.  This  contracting  procedure  should  be  used  by  the  government  on 
a trial  basis. 


MAINTENANCE 

47.  ***  As  recommended  earlier,  institute  a cost  accounting  system  that  will 
afford  visibility  of  the  maintenance  process  and  make  possible  realistic  cost 
comparisons  between  military  and  industrial  maintenance. 

48.  **  Provide  separate  accounts  for  functions  other  than  maintenance,  such  as 
the  use  of  continental  US  maintenance  billets  to  facilitate  the  rotation  of 
military  personnel  not  involved  in  maintenance,  or  for  personnel  in  training 
from  overseas  and  shipboard  billets. 

*49  ***  Establish  alternative  sources  of  maintenance,  including  the  maximum 

feasible  amount  of  contractor  maintenance,  to  foster  competition  and  resul- 
tant efficiency  in  the  maintenance  process  and  to  ensure  the  proper  utiliza- 
tion of  scarce  military  personnel  in  the  present  zero-draft  environment. 

*50.  **  Intensify  efforts  to  reduce  field  maintenance  by  shifting  complex  tasks 

from  the  organizational  and  intermediate  levels  to  the  depots,  taking  due 
account  of  increased  turnaround  time  and  transportation  problems. 

MAINTENANCE  TRAINING 

51.  ***  Develop  fully  proved uralized  job  performance  aids  for  use  in  routine 
maintenance  of  new  weapon  systems  and  for  selected  tasks  in  high-maintenance 
portions  of  existing  systems. 

52.  ***  Selectively,  on  a trial  basis,  reorient  the  training  sequence  for  electronic 
technicians  so  as  to  provide  first  the  specific  training  they  require  to  perform 
maintenance  tasks  by  using  procedural ized  aids  during  their  initial  enlistments. 

53.  Increase  research  on  job  performance  aids  and  on  job-oriented  training  to 
enable  the  utilization  of  personnel  of  lower  ability  levels  and  to  enhance 
learning  on  the  job.  Apply  the  results  in  selected  training  programs. 

WARRANTIES 

*54.  ***  Extend  the  application  of  long-term  contractor  maintenance  warranties 

to  military  electronics  procurements. 

***  Make  known  the  intention  to  contract  for  maintenance  warranties  on 
production  equipment  at  the  lime  development  is  initiated,  so  that  the  con- 
tractor will  design  to  minimize  total  costs  of  production  and  warranty  main- 
tenance. 
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56.  ***  Establish  a warranty  review  group  within  OSD  to  monitor  results  of  trial 

applications,  to  determine  desirable  warranty  contractual  formats,  and  to  refine 
the  categories  of  equipments  to  which  warranties  are  most  applicable  and  for 
which  warranties  are  most  effective. 

*57.  ***  Initially,  apply  long-term  contractor  maintenance  warranties  to  equipments 

whose  failed  units  can  be  replaced  in  the  field  and  conveniently  returned  to 
the  contractor’s  plant  or  base  for  repair,  or  to  which  the  contractor  can  have 
ready  access  for  field  repair,  such  as  airborne  communications,  navigation,  and 
identification  equipment;  modular  radars;  vehicular  communication  sets; 
complex  manpaek  equipment  such  as  LORAN  C/D;  forward-looking  infrared 
(FLIR)  systems;  and  domestic  communication,  data  processing,  and  radar 
installations. 

DESIGN  EVOLUTION  AND  CONFIGURATION  MANAGEMENT 

5X.  ***  The  recently  promulgated  DoD  regulation  on  configuration  management 

(NAVMATINST  4 130.1  A.  for  the  Navy)  should  be  adopted  with  the  follow- 
ing modifications:  (a)  specifically  permit  consideration  of  changes  that  are  of 
benefit  to  the  contractor  and  not  detrimental  to  the  government:  (hi  establish 
two  product  baselines,  the  first  (tentative)  one  at  the  end  of  full-scale  develop- 
ment, and  the  second  (final)  one  when  the  design  has  been  adequately  stabil- 
ized; and  (c)  permit  internal  equipment  changes  that  do  not  affect  form,  fit 
(compatibility  and  interfaces),  function,  price,  or  delivery  to  be  classified 
Class  II  (defined  in  the  regulation)  in  order  to  facilitate  the  change  approval 
process  until  the  "final"  product  baseline  is  invoked  by  the  government. 

59.  ***  The  government  should  defer  invocation  of  the  final  product  baseline,  as 

applicable  to  electronic  equipment,  until  field  reliability  objectives  have  been 
achieved,  or.  in  the  case  of  equipment  under  contract  maintenance  warranty, 
until  the  warranty  period  is  about  to  end  and  the  government  is  about  to  take 
over  maintenance  from  the  warrantor. 

*60.  ***  The  government  should  defer  full  spares  stocking  until  after  the  final 

product  baseline  is  invoked. 

PROJECT  MANAGEMENT 

o I . ***  Use  the  multiprogram  project  office  ("basket"  SPO)  structure  for  all 

independent  electronic  subsystem  developments  where  a number  of  related 
or  similar  developments  can  be  grouped  under  one  perpetual  project  manager 
(PM)  to  provide  a PM  of  higher  rank  and  greater  authority,  better  project 
office  personnel,  more  responsive  support  from  functional  groups,  and  more 
tradeoff  flexibility. 

o3  **4  Provide  nniltiprogram  project  offices  with  sufficient  flexibility  in  the  use 
of  available  R&D  funds  to  allow  the  necessary  tradeoffs  by  the  PM  in  the 
development,  operational  test  and  evaluation,  and  low-rate  initial  production 
phases. 
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63.  **  Arrange  for  the  project  manager  or  prospective  project  manager  to  partici- 

pate in  drafting  the  operational  requirements  before  developing  specifications 
for  subsystems  under  his  jurisdiction. 

64:  **  Make  available  to  system  project  managers  catalogs  of  available  electronic 

equipment  that  show  current  price  and  reliability  figures  as  well  as  technical 
descriptions. 


STANDARDIZATION  AND  SPECIFICATIONS 

(■>5.  ***  l)ol)  should  establish  an  Electronic  Standards  Panel  having  responsibility 

and  authority  to  act  on  recommendations  66  through  76,  following 

66.  ***  Promulgate  policy  requiring  that  the  services  include  electrical,  mechani- 
cal. and  environmental  interface  specifications  in  specifications  for  electronic 
equipment. 

67.  **  Promulgate  policy  requiring  that  the  services  take  steps  toward  assuring 
that  new  electronic  equipments  that  are  likely  to  replace  older  equipments 
in  aircraft,  ground  vehicles,  and  other  platforms  will  be  made  electrically, 
mechanically,  and  environmentally  interchangeable  with  the  older  equipments, 
of  similar  types,  so  that  the  new  equipments  can  be  substituted  for  the  old 
without  costly  installation  modification. 

68.  ***  Promulgate  policy  requiring  that  equipments,  subsystems,  or  systems  of 
similar  types  be  developed  to  the  same  interface  specifications,  so  that  they 
may  be  interchanged. 

L (yg  **  Promulgate  specific  interface  standards  for  classes  of  equipment  used  by 
more  than  one  service. 

70.  **  Establish  and  promulgate  standards  for  the  thermal,  atmospheric,  vibration, 
shock,  mounting,  shielding,  and  power-source  environments  to  be  provided  by 
aircraft,  ships,  and  vehicles  in  which  electronic  equipment  is  to  be  installed. 
Ibis  should  include  standards  for  benign-environment  enclosures  wherever 
they  are  feasible  ami  cost-effective. 

71.  **  With  the  concurrence  of  and  to  the  extent  authorized  by  the  Military 
Communications  Electronics  Board,  establish  and  promulgate  standards  for 
the  signals  to  be  transmitted  or  interchanged  in  cooperative  systems,  such 
as  communications,  navigation,  and  identification  systems. 

72.  **  Review  service  forecasts  of  electronic  equipment  needs  in  order  to  deter- 
mine those  types  and  classes  to  which  uniform  standards  should  be  applied, 
and  act  to  ensure  that  the  standards  are  applied. 

73.  ***  Establish  and  promulgate  DoD  standards  for  the  multiplexing  and  inter 
change  of  digital  data  among  electronic  equipments  within  ships  and  aircraft 

74.  **  Promulgate  policy  designed  to  ensure  maximum  compatibility  of  military 
standards  with  commercial  practices. 
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**  Review  existing  standards  and  specifications  for  parts,  materials,  finishes, 
processes,  and  other  aspects  of  the  internal  design  of  military  electronics  to 
determine  which  should  be  (a)  strictly  enforced,  (b)  subject  to  the  substitu- 
tion of  the  contractor-validated  alternative,  (c)  regarded  as  advisory  only,  or 
(d)  revoked.  The  several  general  design  specifications  used  in  most  electronics 
procurements  (eg.  MIL-E- 16400,  M1L-H-5400.  MIL-I-983)  should  receive 
particular  earlv  attention.  * 

t * 

76.  **  Issue  up-to-date  guidance  on  military  utilization  of  standard  commercial 
LSI  and  MSI  items,  with  particular  attention  to  the  need  for  multiple  sources 
and  avoidance  of  military  unique  designs. 

- 

77.  **  ToD  Directive  4120.3  (DoD  Standardization  Program)  can  be  the  vehicle 
for  the  establishment  of  an  effective  electronic  standards  organization.  In 
order  to  accomplish  this,  the  Defense  Material  Specifications  and  Standards 
Board  should,  under  paragraph  VII  B2  of  the  Directive,  recommend  the  es- 
tablishment of  an  Electronic  Standards  Panel  (ESP)  with  the  authority  and 
responsibility  to  promulgate  multiservice  electronic  standards  and  promote 
the  cause  of  standardization  of  electronic  equipments,  subsystems,  and  sys- 
tems, both  single-service  and  multiservice.  The  ESP  should  be  given  the 
further  authority  to  establish  continuing  (as  opposed  to  ad  hoc)  committees, 
to  which  may  be  delegated  segments  of  the  authority  and  responsibility  of 
the  ESP.  Once  established,  the  ESP  should  organize  to  undertake  formulation 
and  promulgation  of  the  policies  recommended  above. 

SOFTWARE 

To  reduce  costs  of  software  in  processors  employing  conventional  general-purpose 
machines:  ] 

*78.  ***  Complete  the  design  of  the  system  and  the  basic  program  structure  in 

substantial  detail  before  making  major  commitments  to  hardware  or  coding. 

*76.  **  Limit  the  aggregation  of  problems  to  be  solved  on  a central  machine:  as 

an  alternative,  decentralize  processing  by  providing  peripheral  special-purpose 
devices  (either  analog  or  digital)  or  separate  peripheral  general-purpose 
machines  to  perform  specific  separable  functions. 

*80.  **  Select  a processor  of  adequate  size  to  permit  underutilizing  the  computer: 

write  highly  modular  programs:  emphasize  structure  and  overall  efficiency 
rather  than  hardware  efficiency  alone. 

*81.  **  Use  rigorous  discipline  in  software  development,  such  as  the  top-down 

structured-programming  approach. 

*82.  **  Use  a standard,  well  established  programming  language  with  which  pro- 

grammers are  thoroughly  familiar.  Use  the  highest-level  language  appropriate 
to  the  task  at  hand,  but  avoid  the  unnecessary  development  of  a unique 

language.  ; 

*83.  **  Defer  coding  until  the  computer  design  is  substantially  complete  and  firm.  i 

except  for  that  necessary  to  verify  hardware-software  design  compatibility. 
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DIGITAL  SYSTEMS  ARCHITECTURE 


*84.  ***  Systems-funct  ion-oriented  processing-hardware  structures  should  be  con- 

sidered as  alternatives  to  the  conventional  centralized  programmable  unipro- 
cessor for  use  in  military  tactical  systems. 

*85.  ***  The  military  processing  problem  should  be  clearly  stated;  the  system 

design  should  be  spelled  out  in  detail;  and  alternate  processor  architectures 
and  designs  should  be  compared  before  a hardware  approach  is  selected. 

*86.  ***  A processor  design  for  each  system  should  be  selected  and  developed 

that  will  minimize  the  combined  costs  of  hardware  and  software;  the  alloca- 
tion of  functions  between  hardware,  software,  and  human  operators  should 
be  conscientiously  worked  out  prior  to  decision. 

*87.  **  Standard  LSI  processing  elements  available  from  more  than  one  source 

should  be  used  to  the  maximum  extent  possible;  development  iof  uniquely 
military  LSI  elements  should  be  minimized. 

88.  **  Military'  laboratories  should  be  encouraged  to  investigate  and  develop 

processor  architectures,  including  federated  architectures,  that  fit  military 
problems  and  are  cost-effective.  Conversely,  their  extensive  efforts  in  the 
programming  of  conventional  uniprocessors  should  be  reduced  to  bring 
overall  programs  into  better  balance. 

*89.  **  Commercially  successful  processors  for  which  software  already  exists 

should  be  considered  for  DoD  applications  wherever  appropriate. 

90.  **  Formats  and  speeds  for  data  interchange  among  sensors,  actuators,  pro- 

cessors. controls,  and  displays  should  be  standardized  across  service  lines  and 
for  as  wide  a variety  of  applications  as  practicable. 

DATA  COSTS 

*91.  Accept  contractor's  data  format  unless  there  is  a demonstrable  advantage  in 
specifying  a government  format. 

*92.  **  Defer  the  ordering  and  delivery  of  contractor  data  until  the  need  is  firmly 

established. 

*93.  **  Delay  procurement  of  spares  provisioning,  technical  manuals,  and  main- 

tenance handbooks  until  the  point  of  design  stabilization  is  identified  and 
reached . 

*94.  **  Scrub  data  requirements  mercilessly  through  the  efforts  of  Data  Require 

ments  Review  Boards  that  include  representation  of  the  project  manager,  the 
user,  and  industry. 

*95.  Where  the  equipment  future  is  uncertain,  buy  options  on  reprocurement  data 
instead  of  the  data  items  themselves. 

96.  (From  ref  2)  Establish  standards  and  criteria  for  estimating  costs  and  benefits 
of  product  data  requirements.  The  need  for  product  data  should  be  determined 
on  the  basis  of  cost-benefit  analyses.  Selective  after-the-fact  reviews  should 
be  used  as  the  basis  for  eliminating  unnecessary  requirements. 
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97.  (From  ret'  2)  Establish  government-wide  criteria  for  management  systems  which 
are  prescribed  for  use  by  contractors,  including  standards  for  determining 
mission-essential  management  data  requirements. 

Following  recommendations  98-101  are  rewordings  of  previous  recommendations 
in  the  context  of  data  costs. 

98.  Use  competing  suppliers  of  interchangeable  equipment  to  reduce  the  need 
for  reprocurement  data. 

*99.  Use  contractor  warranties  and  maintenance  to  reduce  the  need  for  technical 
and  maintenance  manuals  and  provisioning  data. 

*100.  Rely  on  competitive  prototyping  and  test  as  a substitute  for  voluminous 

in-process  validation  data  (and  as  a substitute  for  myriad  detailed  specifications). 

*101.  As  an  alternative  to  formal  and  highly  detailed  reprocurement  drawings  and 

specifications,  require  less-formal  drawings  and  encourage  more-informal  infor- 
mation transfer.  For  reprocurement  data,  pay  a fixed  amount  for  the  drawings 
plus  a fixed  amount  for  each  equipment  successfully  delivered  by  the  second 
sou  rce. 


GENERAL  PROCUREMENT  CONSIDERATIONS 

102.  (From  ref  5)  To  provide  an  open  environment  in  which  recommended  changes 
can  take  place,  the  hierarchy  of  DoD  program  management  structures  should 
be  realigned  and  simplified. 

103.  (From  ref  2)  Require  the  use  of  formal  advertising  when  the  number  of 
sources,  the  existence  of  adequate  specifications,  and  other  conditions  justify 
such  use.  Authorize  the  use  of  competitive  negotiation  methods  as  an  accep- 
table and  efficient  alternative  to  formal  advertising.  Require  that  the  procure- 
ment file  disclose  the  reasons  for  using  competitive  methods  other  than  lovmal 
advertising,  in  procurements  over  SI 0 000  or  such  other  figure  as  may  be 
established  for  small-purchase  procurers.  Repeal  statutory  provisions  incon- 
sistent with  these  proposed  requirements. 

104.  (From  ref  5)  The  important  role  of  cost-plus-fixed-fee  contracts  should  con- 
tinue for  development  and  prototype  contracts  where  effective  fixed-price 
competition  cannot  be  achieved  without  the  addition  of  large  contingency 
factors. 

105.  (From  ref  2)  The  procurement  of  professional  services  should  be  accomplished, 
so  far  as  practicable,  by  using  competitive  proposal  and  negotiation  procedures 
which  take  into  account  the  technical  competence  ol  the  proposers,  the  pro 
posed  concept  of  the  end  product,  and  the  estimated  cost  of  the  protect, 
including  fee.  The  primary  factors  in  the  selection  process  should  be  the 
professional  competence  of  those  who  will  do  the  work,  and  the  relative 
merits  of  proposals  for  the  end  product,  including  cost,  sought  by  the  gov 
eminent.  The  fee  to  be  charged  should  not  be  the  dominant  factor  in  con 
trading  for  professional  services. 
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106.  (From  ref  2)  Provide  through  legislation  that  it  is  national  policy  to  rely  on 
private  enterprise  for  needed  goods  and  services,  to  the  maximum  extent 
feasible  and  within  the  framework  of  procurement  at  reasonable  prices. 
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APPENDIX  C:  CONTRIBUTING  ORGANIZATIONS  (WASHINGTON.  DC 
UNLESS  OTHERWISE  NOTED) 


Note:  Numerous  (over  800)  individuals  of  the  following  organizations  have 
been  contacted:  specific  names  can  be  provided  upon  request. 

GOVERNMENT  ORGANIZATIONS 

1.  Air  Force  Systems  Command 

2.  Army  Air  Mobility  Command.  Moffett  Field.  CA 

3.  Army  Electronics  Command.  Ft  Monmouth.  NJ 

4.  Army  Mobility  Equipment  Research  and  Development  Center.  Ft  Belvoir.  VA 
8.  Assistant  Secretary  of  Defense  (comptroller) 

6.  Assistant  Secretary  of  Defense  (Installations  & Logistics) 

7.  Assistant  Secretary  of  the  Navy  (Installations  & Logistics) 

8.  Aviation  Supply  Office.  Philadelphia,  PA 

l).  Commander.  Naval  Surface  Forces.  US  Pacific  Fleet.  San  Diego.  CA 

10.  Defense  Advanced  Research  Projects  Agency 

1 1.  Defense  Electronics  Supply  Center.  Dayton,  Oil 

1 2.  Defense  Material  Specifications  and  Standards  Office.  Alexandria.  VA 

13.  Defense  Science  Board 

14.  Defense  Supply  Agency.  Alexandria.  VA 

15.  Director.  Defense  Research  and  Engineering 

16.  Federal  Aviation  Agency 

17.  Federal  Communications  Commission 

18.  Fleet  Maintenance  Support  Office.  Mechanicsburg.  PA 

1 9.  Headquarters.  Naval  Material  Command 

20.  Headquarters.  US  Air  Force 

2 1 Headquarters.  US  Coast  Guard 

22.  Headquarters.  US  Marine  Corps 

23.  Joint  Tactical  Communications  Office  (TRI-LAO) 

24.  Marine  Corps  Development  & Education  Command.  Quantico.  VA 

25.  National  Maritime  Administration 

2(>  National  Security  Agency.  Ft  Meade.  Ml) 

27.  Naval  Air  Development  Center.  Warminster.  PA 

28.  Naval  Air  Rework  Facility,  North  Island  NAS.  CA 
2‘L  Naval  Air  Systems  Command 

30.  Naval  Ammunition  Depot.  Crane.  IN 
31  Naval  Avionics  Facility.  Indianapolis.  IN 


C-l 


! 


32.  Naval  Electronic  Systems  Command 


33.  Naval  Missile  Center.  China  Lake.  C A 

34.  Naval  Postgraduate  School.  Monterey.  C A 

35.  Naval  Research  Laboratory 

36.  Naval  Sea  Systems  Command 

37.  Naval  Ship  Engineering  Center.  Hyattsville,  MD 

38.  Naval  Ship  Engineering  Center,  Norfolk  Div.  Norfolk,  VA 

34.  Naval  Ship  Research  and  Development  Center.  Corderock.  MD 

40.  Naval  Supply  Systems  Command 

4 1 . Naval  Surface  Weapons  Centers.  Dahlgreen  and  White  Oak,  MD 

42.  Naval  Undersea  Center.  San  Diego,  CA 

43.  Office  of  the  Chief  of  Naval  Operations 

44.  Office  of  Naval  Research 

45.  Rome  Air  Development  Center.  Rome,  NY 

46.  Ship  Parts  Control  Center,  Mechanicsburg,  PA 

47.  Small  Business  Administration 

NONGOVERNMENT  ORGANIZATIONS 

1.  Applied  Physics  Laboratory,  John  Hopkins  University.  Laurel,  MD 

2.  ARINC  Research  C'orp.  Annapolis.  MD 

3.  Battelle  Columbus  Laboratories.  Columbus.  Oil 

4.  Bell  Telephone  Laboratories.  Whippany.  NJ 

5.  Boeing  Aerospace  Co,  Seattle  WA 

6.  Collins  Radio  Co,  Dallas.  TX 

7.  Continental  Testing  Laboratories.  Orlando.  FL 

8.  Chrysler  Corp,  Detroit.  Ml.  and  Huntsville.  AL 
l>.  Cubic  Corp,  San  Diego,  CA 

10.  Datatron  Inc.  Microelectronics  Testing  Laboratories.  Santa  Ana.  CA 
I I Daystrom  Industries.  Heath  Co.  Benton  Harbor,  Ml 

12.  DC  A Reliability  Laboratory.  Mt  View.  CA 

13.  Electronic  Arrays.  Inc.  Mt  View.  C'A 

14.  Electronic  Industries  Association,  Washington.  DC' 

15.  Fairchild  Camera  and  Instrument  Corp.  Semiconductor  Group.  Mt  View.  CA 
l(v  Ford  Motor  Co.  Philco-Ford  Div.  Landsdale.  PA 

17.  General  Dynamics  Corp,  Convair  Aerospace  Div.  San  Diego.  CA 

18.  General  Dynamics  Corp.  Electric  Boat  Div.  Groton,  CT 


I1).  General  Dynamics  C'orp,  Electronics  Div.  San  Diego,  C A 
20.  General  Dynamics  C'orp.  Stromberg-Carlson  Div.  Rochester,  NY 
2 I . General  Electric  Go.  Schenectady.  NY 
2Y  General  Motors  C'orp,  Chevrolet  Div.  Detroit.  MI 

23.  General  Motors  C’orp.  Delco  Div.  Kokomo,  IN 

24.  Globe-Union.  Inc.  Milwaukee.  Wl 

25.  Harris-lntertype  C'orp.  Harris  Semiconductor  Div.  Melbourne.  EL 

26.  Hewlett-Packard  Co.  Palo  Alto,  CA 

27.  Honeywell.  Boston  Computer  Op.  Billerica.  MA 

28.  Honeywell.  Aerospace  Div.  St  Petersburg.  FL 

2d.  Hughes  Aircraft  Co.  Microelectronic  Products  Div,  Newport  Beach.  CA 

30.  Hughes  Aircraft  Co.  Data  Systems  Div.  Culver  C'ity.  CA 

31.  IEEE  Engineering  Management  Society.  New  York.  NY 

32.  Integrated  Circuit  Engineering  Corp,  Phoenix,  AZ 

33.  INTEL  C'orp,  Santa  Clara,  CA 

34.  Intersil.  Inc,  Cupertino.  CA 

35.  Institute  for  Defense  Analyses.  Arlington.  VA 

36.  1 1 I K C'orp.  Applied  Technology  Div.  Sunnyvale.  CA 

37.  ITT  Avionics  Div.  Nutley.  NJ 

38.  ITT  Defense  Communications  Div.  Nutlex  . NJ 
3d.  ITT  Federal  Electric  Div.  Paramus,  N.I 

40.  I ll'  Semiconductor  Div,  West  Palm  Beach.  IT 
4 I . Lear  Siegler.  Inc.  Santa  Monica.  CA 

42.  Lockheed  Missile  & Space  Div,  Sunnyvale.  CA 

43.  LTV  Aerospace  C'orp,  Vought  Systems  Div.  Dallas.  I \ 

44.  Martin  Marietta  Corp.  Orlando  Div.  Orlando.  I I 

45.  McDonnell  Douglas  Corp,  St  Louis.  MO 

46.  3-M  Company.  Minneapolis.  MN 

47.  Motorola.  Consumer  Product  Div.  Chicago.  II 

48.  Motorola.  Government  Electronic  Div.  Scottsdale.  A/ 

4d.  Motorola.  Semiconductor  Products  Div.  Phoenix.  A/ 

50.  National  Semiconductor  C’orp.  Santa  Clara.  CA 

51.  National  Steel  and  Shipbuilding  Co.  San  Diego.  C A 

52.  NCR  C'orp.  I Cl  Div.  St  Petersburg.  I I 

53.  Northrop  C'orp  Electronics  Div.  Palos  Verdes.  CA 
54  Pacific  Southwest  Airlines  San  Diego.  CA 

55,  Pacific  Telephone  and  Telegraph  Co.  San  Diego  Office.  San  Diego.  C \ 
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56.  Pan  American  World  Airways,  Jamaica,  NY 

57.  Polaroid  Corp.  Cambridge.  MA 

58.  Rand  Corp,  Economics  Dept,  Santa  Monica.  CA 

59.  Raytheon  Equipment  Development  Laboratory,  Sudbury.  MA 

60.  Raytheon  Missile  Systems  Div.  Bedford,  MA 

61 . RCA  Home  Products  Div.  Indianapolis,  IN 

62.  RCA  Missile  and  Surface  Radar  Div.  Morristown,  NJ 

63.  RCA  Solid  State  Div.  Sommerville.  NJ 

64.  Rockwell  International.  Autoneties  Div.  Anaheim,  CA 

65.  Sea  Service  Tankers.  Inc,  San  Diego  Representative,  San  Diego,  CA 

66.  Sears.  Roebuck  and  Co,  Chicago,  IL 

67.  Science  Applications,  Inc.  Sunnyvale,  CA 

68.  Semiconductor  and  Component  Testing  Laboratory.  Dallas.  TX 

69.  Sperry  Rand  Research  Center.  Sudbury,  MA 

70.  TEC  Services,  Inc,  Dallas.  TX 

71 . Texas  Instruments.  Dallas.  TX 

72.  Todd  Shipbuilding  Co,  San  Pedro,  ('A 

73.  TRW  Systems.  Redondo  Beach.  CA 

74.  UCLA  Graduate  School  of  Management.  Los  Angeles.  CA 

75.  University  of  Texas,  School  of  Library  and  Information  Sciences.  Austin.  I \ 

76.  Value  Engineering  Co,  West  Long  Branch.  NJ 

77.  Wavetec  Corp.  San  Diego.  CA 

78.  Western  Electric  Co.  Chicago.  1L 
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